* Outreachy - Guix Data Service: implementing basic json output for derivation comparison page @ 2021-04-14 19:48 Luciana Lima Brito 2021-04-15 8:46 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-14 19:48 UTC (permalink / raw) To: guix-devel [-- Attachment #1: Type: text/plain, Size: 270 bytes --] Hi, I implemented a basic json output for the derivation comparison page, for my first contribution as an Outreachy applicant. The patch for the code I've changed is attached. I'm waiting your reviews :) -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: controller.patch --] [-- Type: text/x-patch, Size: 6884 bytes --] diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..b7788cb 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -584,19 +584,115 @@ (derivation-differences-data conn base-derivation target-derivation))))) - (case (most-appropriate-mime-type - '(application/json text/html) - mime-types) - ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) - (else - (render-html - #:sxml (compare/derivation - query-parameters - data) - #:extra-headers http-headers-for-unchanging-content))))))) + (let ((outputs (assq-ref data 'outputs)) + (inputs (assq-ref data 'inputs)) + (sources (assq-ref data 'sources)) + (system (assq-ref data 'system)) + (builder (assq-ref data 'builder)) + (args (assq-ref data 'arguments)) + (environment-variables (assq-ref data 'environment-variables)) + (get-derivation-data + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t")))))) + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output))))) + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '()))))) + + (let ((base-system (assq-ref system 'base)) + (target-system (assq-ref system 'target)) + (common-system (assq-ref system 'common)) + + (base-builder (assq-ref builder 'base)) + (target-builder (assq-ref builder 'target)) + (common-builder (assq-ref builder 'common)) + + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target)) + (common-args (assq-ref args 'common))) + + (let ((matched-outputs (append-map get-derivation-data + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + (matched-inputs (append-map get-derivation-data + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + (matched-sources (append-map get-derivation-data + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common))))) + (case (most-appropriate-mime-type + '(application/json text/html) + mime-types) + ((application/json) + (render-json + `((revision + . ((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))))) + (outputs + . ,((lambda (l) (cond + ((= (length l) 3) `((base . ,(first l)) + (target . ,(second l)) + (common . ,(third l)))) + ((= (length l) 2) `((base . ,(first l)) + (target . ,(second l)))) + (else `((common . ,(first l)))))) + matched-outputs)) + (inputs + . ((base . ,(first matched-inputs)) + (target . ,(second matched-inputs)))) + (source + . ((base . ,(first matched-sources)) + (target . ,(second matched-sources)) + (common . ,(third matched-sources)))) + (system + . ((common . ,common-system))) + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,environment-variables)) + #:extra-headers http-headers-for-unchanging-content)) + (else + (render-html + #:sxml (compare/derivation + query-parameters + data) + #:extra-headers http-headers-for-unchanging-content)))))))))) (define (render-compare/package-derivations mime-types query-parameters) ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-14 19:48 Outreachy - Guix Data Service: implementing basic json output for derivation comparison page Luciana Lima Brito @ 2021-04-15 8:46 ` Christopher Baines 2021-04-15 16:09 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-15 8:46 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 8713 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > I implemented a basic json output for the derivation comparison page, > for my first contribution as an Outreachy applicant. > > The patch for the code I've changed is attached. > I'm waiting your reviews :) Hi Luciana, I'm not quite sure how to apply this, I'd suggest using git format-patch to generate the file next time as I think there would normally be some metadata along with the diff. Looking at the diff though: > diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm > index a6aa198..b7788cb 100644 > --- a/guix-data-service/web/compare/controller.scm > +++ b/guix-data-service/web/compare/controller.scm > @@ -584,19 +584,115 @@ > (derivation-differences-data conn > base-derivation > target-derivation))))) > - (case (most-appropriate-mime-type > - '(application/json text/html) > - mime-types) > - ((application/json) > - (render-json > - '((error . "unimplemented")) ; TODO > - #:extra-headers http-headers-for-unchanging-content)) > - (else > - (render-html > - #:sxml (compare/derivation > - query-parameters > - data) > - #:extra-headers http-headers-for-unchanging-content))))))) > + (let ((outputs (assq-ref data 'outputs)) > + (inputs (assq-ref data 'inputs)) > + (sources (assq-ref data 'sources)) > + (system (assq-ref data 'system)) > + (builder (assq-ref data 'builder)) > + (args (assq-ref data 'arguments)) > + (environment-variables (assq-ref data 'environment-variables)) > + (get-derivation-data > + (lambda (items) > + (map > + (match-lambda > + ((name path hash-alg hash recursive) > + `(,@(if (null? name) > + '() > + `((name . ,name))) > + ,@(if (null? path) > + '() > + `((path . ,path)) > + ) > + ,@(if (or (null? hash-alg) (not (string? hash-alg))) > + '() > + `((hash-algorithm . ,hash-alg)) > + ) > + ,@(if (or (null? hash) (not (string? hash))) > + '() > + `((hash . ,hash)) > + ) > + ,@(if (null? recursive) > + '() > + `((recursive . ,(string=? recursive "t")))))) > + ((derivation output) > + `(,@(if (null? derivation) > + '() > + `((derivation . ,derivation))) > + ,@(if (null? output) > + '() > + `((output . ,output))))) > + ((derivation) > + `(,@(if (null? derivation) > + '() > + `((derivation . ,derivation)))))) > + (or items '()))))) > + > + (let ((base-system (assq-ref system 'base)) > + (target-system (assq-ref system 'target)) > + (common-system (assq-ref system 'common)) > + > + (base-builder (assq-ref builder 'base)) > + (target-builder (assq-ref builder 'target)) > + (common-builder (assq-ref builder 'common)) > + > + (base-args (assq-ref args 'base)) > + (target-args (assq-ref args 'target)) > + (common-args (assq-ref args 'common))) > + > + (let ((matched-outputs (append-map get-derivation-data > + (list (assq-ref outputs 'base) > + (assq-ref outputs 'target) > + (assq-ref outputs 'common)))) > + (matched-inputs (append-map get-derivation-data > + (list (assq-ref inputs 'base) > + (assq-ref inputs 'target)))) > + (matched-sources (append-map get-derivation-data > + (list (assq-ref sources 'base) > + (assq-ref sources 'target) > + (assq-ref sources 'common))))) I would consider whether it's useful to have all these let blocks, and whether here is the right place for them. Taking a binding like outputs, it's only used in a later let. You can do something like this (with let*) to remove the need to have multiple let blocks. (let* ((outputs (assq-ref data 'outputs)) (matched-outputs (append-map get-derivation-data (list (assq-ref outputs 'base) (assq-ref outputs 'target) (assq-ref outputs 'common)))) Also, since matched-outputs is only used when rendering the JSON, I'd move all the bindings that are only used for the JSON output within that part of the case statement, so that it's clearer that they only apply to that bit of the code. Does that make sense? > + (case (most-appropriate-mime-type > + '(application/json text/html) > + mime-types) > + ((application/json) > + (render-json > + `((revision I'm not sure what revision here referrs to. > + . ((base > + . ((derivation . ,base-derivation))) > + (target > + . ((derivation . ,target-derivation))))) > + (outputs > + . ,((lambda (l) (cond > + ((= (length l) 3) `((base . ,(first l)) > + (target . ,(second l)) > + (common . ,(third l)))) > + ((= (length l) 2) `((base . ,(first l)) > + (target . ,(second l)))) > + (else `((common . ,(first l)))))) > + matched-outputs)) > + (inputs > + . ((base . ,(first matched-inputs)) > + (target . ,(second matched-inputs)))) > + (source > + . ((base . ,(first matched-sources)) > + (target . ,(second matched-sources)) > + (common . ,(third matched-sources)))) > + (system > + . ((common . ,common-system))) > + (builder-and-arguments > + . ((builder . ,common-builder) > + (arguments > + . ((base . ,(list->vector > + base-args)) > + (target . ,(list->vector > + target-args)))))) > + (environment-variables . ,environment-variables)) > + #:extra-headers http-headers-for-unchanging-content)) > + (else > + (render-html > + #:sxml (compare/derivation > + query-parameters > + data) > + #:extra-headers http-headers-for-unchanging-content)))))))))) > > (define (render-compare/package-derivations mime-types > query-parameters) I hope that helps, just let me know if you have any questions, Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-15 8:46 ` Christopher Baines @ 2021-04-15 16:09 ` Luciana Lima Brito 2021-04-15 23:19 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-15 16:09 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1517 bytes --] On Thu, 15 Apr 2021 09:46:12 +0100 Christopher Baines <mail@cbaines.net> wrote: Hi, > I'm not quite sure how to apply this, I'd suggest using git > format-patch to generate the file next time as I think there would > normally be some metadata along with the diff. I tried using git format-patch and I got 7 patches from my 7 commits, then I generate a single patch output, which is attached. The last commit before my modifications is this: 410f58cb43f083623885a430700c6818a187cadc > I would consider whether it's useful to have all these let blocks, and > whether here is the right place for them. > Taking a binding like outputs, it's only used in a later let. You can > do something like this (with let*) to remove the need to have > multiple let blocks. > Also, since matched-outputs is only used when rendering the JSON, I'd > move all the bindings that are only used for the JSON output within > that part of the case statement, so that it's clearer that they only > apply to that bit of the code. > > Does that make sense? I did it, I used the let* and this helped a lot. I also moved everything into the case branch of the json. > I'm not sure what revision here referrs to. It was a placeholder, but now I removed it. > I hope that helps, just let me know if you have any questions, The function get-derivation-data does not depend on anything, don't you think it goes better in another place outside render-compare/derivation? -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: controller-0415.patch --] [-- Type: text/x-patch, Size: 65186 bytes --] From c7af970c677a8d97cb4841a748e343c03e0bc886 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH 1/7] Include base-derivation and target-derivation on json of render-compare/derivation --- guix-data-service/web/compare/controller.scm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..355ce03 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -589,7 +589,12 @@ mime-types) ((application/json) (render-json - '((error . "unimplemented")) ; TODO + `((revision + . ((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation)))))) + ;'((error . "unimplemented")) ; TODO #:extra-headers http-headers-for-unchanging-content)) (else (render-html -- 2.30.2 From d91856c6b4a0e998288b2291f9a9f420f9916dc4 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Mon, 12 Apr 2021 11:05:20 -0300 Subject: [PATCH 2/7] Include base and target outputs to json of render-compare/derivation --- guix-data-service/web/compare/controller.scm | 127 ++++++++++++++++--- 1 file changed, 109 insertions(+), 18 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index 355ce03..2f54310 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -584,24 +584,115 @@ (derivation-differences-data conn base-derivation target-derivation))))) - (case (most-appropriate-mime-type - '(application/json text/html) - mime-types) - ((application/json) - (render-json - `((revision - . ((base - . ((derivation . ,base-derivation))) - (target - . ((derivation . ,target-derivation)))))) - ;'((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) - (else - (render-html - #:sxml (compare/derivation - query-parameters - data) - #:extra-headers http-headers-for-unchanging-content))))))) + + (let ((outputs (assq-ref data 'outputs)) + (inputs (assq-ref data 'inputs)) + (sources (assq-ref data 'sources)) + (system (assq-ref data 'system)) + (build-and-args (assq-ref data 'build-and-args)) + (envronment-and-variables (assq-ref data 'envronment-and-variables)) + ) + (let ((base-outputs (assq-ref outputs 'base)) + (target-outputs (assq-ref outputs 'target)) + + (base-inputs (assq-ref inputs 'base)) + (target-inputs (assq-ref inputs 'target)) + + (base-sources (assq-ref sources 'base)) + (target-sources (assq-ref sources 'target)) + + (base-system (assq-ref system 'base)) + (target-system (assq-ref system 'target)) + + (base-build-and-args (assq-ref build-and-args 'base)) + (target-build-and-args (assq-ref build-and-args 'target)) + + (base-environment-and-variables (assq-ref envronment-and-variables 'base)) + (target-environment-and-variables (assq-ref envronment-and-variables 'target)) + + ) + + ;(let-values + ; (((base-derivation-output target-derivation-output) + ; (assq-ref data 'outputs))) + ; (let ((derivation-changes + ; (package-derivation-data-changes names-and-versions + ; base-packages-vhash + ; target-packages-vhash))) + + (case (most-appropriate-mime-type + '(application/json text/html) + mime-types) + ((application/json) + (render-json + `((revision + . ((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))))) + (outputs . ((base . ,(list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (null? hash-alg) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (null? hash) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,recursive)) + ) + ;(change . ,change) + ))) + base-outputs))) + + (target . ,(list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (null? hash-alg) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (null? hash) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,recursive)) + ) + ;(change . ,change) + ))) + target-outputs)))) + + ) + ) + #:extra-headers http-headers-for-unchanging-content)) + (else + (render-html + #:sxml (compare/derivation + query-parameters + data) + #:extra-headers http-headers-for-unchanging-content))))))))) (define (render-compare/package-derivations mime-types query-parameters) -- 2.30.2 From 3810e457e75a24ae1c1582a35956c6bc9e37c614 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Mon, 12 Apr 2021 15:52:17 -0300 Subject: [PATCH 3/7] Include base and target outputs to json in same anonymous function of render-compare/derivation --- guix-data-service/web/compare/controller.scm | 84 +++++++------------- 1 file changed, 30 insertions(+), 54 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index 2f54310..7fa2fc0 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -630,62 +630,38 @@ . ((derivation . ,base-derivation))) (target . ((derivation . ,target-derivation))))) - (outputs . ((base . ,(list->vector - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (null? hash-alg) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (null? hash) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,recursive)) - ) + (outputs . ,(list->vector + (append-map + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (null? hash-alg) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (null? hash) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,recursive)) + ) ;(change . ,change) - ))) - base-outputs))) + ))) + (or items '()))) - (target . ,(list->vector - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (null? hash-alg) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (null? hash) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,recursive)) - ) - ;(change . ,change) - ))) - target-outputs)))) - - ) - ) + (list base-outputs + target-outputs)))) + );revision #:extra-headers http-headers-for-unchanging-content)) (else (render-html -- 2.30.2 From aa5292177bbe70564306cebdebff417754b807b6 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Wed, 14 Apr 2021 14:04:21 -0300 Subject: [PATCH 4/7] Fix rendering for outputs on json from render-compare/derivation --- guix-data-service/web/compare/controller.scm | 231 +++++++++++++------ 1 file changed, 165 insertions(+), 66 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index 7fa2fc0..dcb0175 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -589,86 +589,185 @@ (inputs (assq-ref data 'inputs)) (sources (assq-ref data 'sources)) (system (assq-ref data 'system)) - (build-and-args (assq-ref data 'build-and-args)) - (envronment-and-variables (assq-ref data 'envronment-and-variables)) + (builder (assq-ref data 'builder)) + (args (assq-ref data 'arguments)) + (environment-variables (assq-ref data 'environment-variables)) ) - (let ((base-outputs (assq-ref outputs 'base)) - (target-outputs (assq-ref outputs 'target)) - + + (let (;(base-outputs (assq-ref outputs 'base)) + ;(target-outputs (assq-ref outputs 'target)) + ;(common-outputs (assq-ref outputs 'common)) + + (matched-outputs (append-map + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t"))))) + )) + (or items '()))) + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + (base-inputs (assq-ref inputs 'base)) (target-inputs (assq-ref inputs 'target)) (base-sources (assq-ref sources 'base)) (target-sources (assq-ref sources 'target)) + (common-sources (assq-ref sources 'common)) (base-system (assq-ref system 'base)) (target-system (assq-ref system 'target)) + (common-system (assq-ref system 'common)) - (base-build-and-args (assq-ref build-and-args 'base)) - (target-build-and-args (assq-ref build-and-args 'target)) + (base-builder (assq-ref builder 'base)) + (target-builder (assq-ref builder 'target)) + (common-builder (assq-ref builder 'common)) - (base-environment-and-variables (assq-ref envronment-and-variables 'base)) - (target-environment-and-variables (assq-ref envronment-and-variables 'target)) - + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target)) + (common-args (assq-ref args 'common)) ) - ;(let-values - ; (((base-derivation-output target-derivation-output) - ; (assq-ref data 'outputs))) - ; (let ((derivation-changes - ; (package-derivation-data-changes names-and-versions - ; base-packages-vhash - ; target-packages-vhash))) - - (case (most-appropriate-mime-type - '(application/json text/html) - mime-types) - ((application/json) - (render-json - `((revision - . ((base - . ((derivation . ,base-derivation))) - (target - . ((derivation . ,target-derivation))))) - (outputs . ,(list->vector - (append-map - (lambda (items) - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (null? hash-alg) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (null? hash) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,recursive)) - ) - ;(change . ,change) - ))) - (or items '()))) + (match environment-variables + ((name . values) + (let ((environment-variables-name name) + (common-environment-variables (assq-ref values 'common)) + (base-environment-variables (assq-ref values 'base)) + (target-environment-variables (assq-ref values 'target)) + ) + + (case (most-appropriate-mime-type + '(application/json text/html) + mime-types) + ((application/json) + (render-json + `((revision + . ((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))))) + ;(outputs . ,(list->vector + ; (append-map + ; (lambda (items) + ; (map + ; (match-lambda + ; ((name path hash-alg hash recursive) + ; `(,@(if (null? name) + ; '() + ; `((name . ,name))) + ; ,@(if (null? path) + ; '() + ; `((path . ,path)) + ; ) + ; ,@(if (or (null? hash-alg) (not (string? hash-alg))) + ; '() + ; `((hash-algorithm . ,hash-alg)) + ; ) + ; ,@(if (or (null? hash) (not (string? hash))) + ; '() + ; `((hash . ,hash)) + ; ) + ; ,@(if (null? recursive) + ; '() + ;if there is other representation for "true" + ;this parse won't work + ; `((recursive . ,(string=? recursive "t"))) + ; ) + ; ))) + ; (or items '())) + ; );inner lambda - (list base-outputs - target-outputs)))) - );revision - #:extra-headers http-headers-for-unchanging-content)) - (else - (render-html - #:sxml (compare/derivation - query-parameters - data) - #:extra-headers http-headers-for-unchanging-content))))))))) + ; (list base-outputs + ; target-outputs + ; common-outputs)))) + + (output + . ,((lambda (x) (cond + ((= (length x) 3) `((base . ,(first x)) + (target . ,(second x)) + (common . ,(third x)))) + ((= (length x) 2) `((base . ,(first x)) + (target . ,(second x)))) + (else `((common . ,(first x)))))) + + matched-outputs)) + + (inputs . ,(list->vector + (append-map + (lambda (items) + (map + (match-lambda + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output)))))) + (or items '()))) + (list base-inputs + target-inputs) + ))) + + (source + . ((base . ,base-sources))) + +; (sources . ,(list->vector +; (append-map +; (lambda (items) +; (map +; (match-lambda +; ((derivation) +; `(,@(if (null? derivation) +; '() +; `((derivation . ,derivation)))))) +; (or items '()))) +; (list base-sources +; target-sources +; common-sources))) +; ) + + (system + . ((common . ,common-system))) + + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + + (environment-variables . ,environment-variables) + + + );revision + #:extra-headers http-headers-for-unchanging-content)) + (else + (render-html + #:sxml (compare/derivation + query-parameters + data) + #:extra-headers http-headers-for-unchanging-content)))))))))))) (define (render-compare/package-derivations mime-types query-parameters) -- 2.30.2 From b61825ca583b1a3d598575f12aa8c6e279ca60ad Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Wed, 14 Apr 2021 15:37:19 -0300 Subject: [PATCH 5/7] Fix rendering for sources on json from remder-compare/derivation --- guix-data-service/web/compare/controller.scm | 169 +++++++------------ 1 file changed, 61 insertions(+), 108 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index dcb0175..bfec00c 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -594,46 +594,46 @@ (environment-variables (assq-ref data 'environment-variables)) ) - (let (;(base-outputs (assq-ref outputs 'base)) - ;(target-outputs (assq-ref outputs 'target)) - ;(common-outputs (assq-ref outputs 'common)) - - (matched-outputs (append-map - (lambda (items) - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (or (null? hash-alg) (not (string? hash-alg))) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (or (null? hash) (not (string? hash))) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,(string=? recursive "t"))))) - )) - (or items '()))) - (list (assq-ref outputs 'base) - (assq-ref outputs 'target) - (assq-ref outputs 'common)))) + (let ((get-derivation-data + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t")))))) + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output))))) + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + )) + ) + (or items '()) + )) + ) - (base-inputs (assq-ref inputs 'base)) - (target-inputs (assq-ref inputs 'target)) - - (base-sources (assq-ref sources 'base)) - (target-sources (assq-ref sources 'target)) - (common-sources (assq-ref sources 'common)) - (base-system (assq-ref system 'base)) (target-system (assq-ref system 'target)) (common-system (assq-ref system 'common)) @@ -647,6 +647,19 @@ (common-args (assq-ref args 'common)) ) + (let ((matched-outputs (append-map get-derivation-data + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + (matched-inputs (append-map get-derivation-data + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + (matched-sources (append-map get-derivation-data + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + ) + (match environment-variables ((name . values) (let ((environment-variables-name name) @@ -665,42 +678,7 @@ . ((derivation . ,base-derivation))) (target . ((derivation . ,target-derivation))))) - ;(outputs . ,(list->vector - ; (append-map - ; (lambda (items) - ; (map - ; (match-lambda - ; ((name path hash-alg hash recursive) - ; `(,@(if (null? name) - ; '() - ; `((name . ,name))) - ; ,@(if (null? path) - ; '() - ; `((path . ,path)) - ; ) - ; ,@(if (or (null? hash-alg) (not (string? hash-alg))) - ; '() - ; `((hash-algorithm . ,hash-alg)) - ; ) - ; ,@(if (or (null? hash) (not (string? hash))) - ; '() - ; `((hash . ,hash)) - ; ) - ; ,@(if (null? recursive) - ; '() - ;if there is other representation for "true" - ;this parse won't work - ; `((recursive . ,(string=? recursive "t"))) - ; ) - ; ))) - ; (or items '())) - ; );inner lambda - - ; (list base-outputs - ; target-outputs - ; common-outputs)))) - - (output + (outputs . ,((lambda (x) (cond ((= (length x) 3) `((base . ,(first x)) (target . ,(second x)) @@ -711,41 +689,16 @@ matched-outputs)) - (inputs . ,(list->vector - (append-map - (lambda (items) - (map - (match-lambda - ((derivation output) - `(,@(if (null? derivation) - '() - `((derivation . ,derivation))) - ,@(if (null? output) - '() - `((output . ,output)))))) - (or items '()))) - (list base-inputs - target-inputs) - ))) + (inputs + . ((base . ,(first matched-inputs)) + (target . ,(second matched-inputs)) + )) (source - . ((base . ,base-sources))) + . ((base . ,(first matched-sources)) + (target . ,(second matched-sources)) + (common . ,(third matched-sources)))) -; (sources . ,(list->vector -; (append-map -; (lambda (items) -; (map -; (match-lambda -; ((derivation) -; `(,@(if (null? derivation) -; '() -; `((derivation . ,derivation)))))) -; (or items '()))) -; (list base-sources -; target-sources -; common-sources))) -; ) - (system . ((common . ,common-system))) @@ -767,7 +720,7 @@ #:sxml (compare/derivation query-parameters data) - #:extra-headers http-headers-for-unchanging-content)))))))))))) + #:extra-headers http-headers-for-unchanging-content))))))))))))) (define (render-compare/package-derivations mime-types query-parameters) -- 2.30.2 From 2c8d95aea7ef3f2950bbb125c995e009f61edf12 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Wed, 14 Apr 2021 16:28:43 -0300 Subject: [PATCH 6/7] Generate a complete json output for render-compare/derivation procedure for the derivation comparison page. --- guix-data-service/web/compare/controller.scm | 202 ++++++++----------- 1 file changed, 87 insertions(+), 115 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index bfec00c..b7788cb 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -584,57 +584,50 @@ (derivation-differences-data conn base-derivation target-derivation))))) - (let ((outputs (assq-ref data 'outputs)) - (inputs (assq-ref data 'inputs)) + (inputs (assq-ref data 'inputs)) (sources (assq-ref data 'sources)) - (system (assq-ref data 'system)) + (system (assq-ref data 'system)) (builder (assq-ref data 'builder)) - (args (assq-ref data 'arguments)) + (args (assq-ref data 'arguments)) (environment-variables (assq-ref data 'environment-variables)) - ) - - (let ((get-derivation-data - (lambda (items) - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (or (null? hash-alg) (not (string? hash-alg))) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (or (null? hash) (not (string? hash))) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,(string=? recursive "t")))))) - ((derivation output) - `(,@(if (null? derivation) - '() - `((derivation . ,derivation))) - ,@(if (null? output) - '() - `((output . ,output))))) - ((derivation) - `(,@(if (null? derivation) - '() - `((derivation . ,derivation))) - )) - ) - (or items '()) - )) - ) - - (base-system (assq-ref system 'base)) + (get-derivation-data + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t")))))) + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output))))) + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '()))))) + + (let ((base-system (assq-ref system 'base)) (target-system (assq-ref system 'target)) (common-system (assq-ref system 'common)) @@ -644,8 +637,7 @@ (base-args (assq-ref args 'base)) (target-args (assq-ref args 'target)) - (common-args (assq-ref args 'common)) - ) + (common-args (assq-ref args 'common))) (let ((matched-outputs (append-map get-derivation-data (list (assq-ref outputs 'base) @@ -655,72 +647,52 @@ (list (assq-ref inputs 'base) (assq-ref inputs 'target)))) (matched-sources (append-map get-derivation-data - (list (assq-ref sources 'base) - (assq-ref sources 'target) - (assq-ref sources 'common)))) - ) - - (match environment-variables - ((name . values) - (let ((environment-variables-name name) - (common-environment-variables (assq-ref values 'common)) - (base-environment-variables (assq-ref values 'base)) - (target-environment-variables (assq-ref values 'target)) - ) - - (case (most-appropriate-mime-type - '(application/json text/html) - mime-types) - ((application/json) - (render-json - `((revision - . ((base - . ((derivation . ,base-derivation))) - (target - . ((derivation . ,target-derivation))))) - (outputs - . ,((lambda (x) (cond - ((= (length x) 3) `((base . ,(first x)) - (target . ,(second x)) - (common . ,(third x)))) - ((= (length x) 2) `((base . ,(first x)) - (target . ,(second x)))) - (else `((common . ,(first x)))))) - - matched-outputs)) - - (inputs - . ((base . ,(first matched-inputs)) - (target . ,(second matched-inputs)) - )) - - (source - . ((base . ,(first matched-sources)) - (target . ,(second matched-sources)) - (common . ,(third matched-sources)))) - - (system - . ((common . ,common-system))) - - (builder-and-arguments - . ((builder . ,common-builder) - (arguments - . ((base . ,(list->vector - base-args)) - (target . ,(list->vector - target-args)))))) - - (environment-variables . ,environment-variables) - - - );revision - #:extra-headers http-headers-for-unchanging-content)) - (else - (render-html - #:sxml (compare/derivation - query-parameters - data) - #:extra-headers http-headers-for-unchanging-content))))))))))))) + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common))))) + (case (most-appropriate-mime-type + '(application/json text/html) + mime-types) + ((application/json) + (render-json + `((revision + . ((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))))) + (outputs + . ,((lambda (l) (cond + ((= (length l) 3) `((base . ,(first l)) + (target . ,(second l)) + (common . ,(third l)))) + ((= (length l) 2) `((base . ,(first l)) + (target . ,(second l)))) + (else `((common . ,(first l)))))) + matched-outputs)) + (inputs + . ((base . ,(first matched-inputs)) + (target . ,(second matched-inputs)))) + (source + . ((base . ,(first matched-sources)) + (target . ,(second matched-sources)) + (common . ,(third matched-sources)))) + (system + . ((common . ,common-system))) + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,environment-variables)) + #:extra-headers http-headers-for-unchanging-content)) + (else + (render-html + #:sxml (compare/derivation + query-parameters + data) + #:extra-headers http-headers-for-unchanging-content)))))))))) (define (render-compare/package-derivations mime-types query-parameters) -- 2.30.2 From e8d99e4a422dc9ee01625e5a3206768ba09a2d54 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Thu, 15 Apr 2021 12:47:46 -0300 Subject: [PATCH 7/7] Improve json output for render-data/derivation --- guix-data-service/web/compare/controller.scm | 213 +++++++++---------- 1 file changed, 103 insertions(+), 110 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index b7788cb..882652a 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -583,116 +583,109 @@ (lambda (conn) (derivation-differences-data conn base-derivation - target-derivation))))) - (let ((outputs (assq-ref data 'outputs)) - (inputs (assq-ref data 'inputs)) - (sources (assq-ref data 'sources)) - (system (assq-ref data 'system)) - (builder (assq-ref data 'builder)) - (args (assq-ref data 'arguments)) - (environment-variables (assq-ref data 'environment-variables)) - (get-derivation-data - (lambda (items) - (map - (match-lambda - ((name path hash-alg hash recursive) - `(,@(if (null? name) - '() - `((name . ,name))) - ,@(if (null? path) - '() - `((path . ,path)) - ) - ,@(if (or (null? hash-alg) (not (string? hash-alg))) - '() - `((hash-algorithm . ,hash-alg)) - ) - ,@(if (or (null? hash) (not (string? hash))) - '() - `((hash . ,hash)) - ) - ,@(if (null? recursive) - '() - `((recursive . ,(string=? recursive "t")))))) - ((derivation output) - `(,@(if (null? derivation) - '() - `((derivation . ,derivation))) - ,@(if (null? output) - '() - `((output . ,output))))) - ((derivation) - `(,@(if (null? derivation) - '() - `((derivation . ,derivation)))))) - (or items '()))))) - - (let ((base-system (assq-ref system 'base)) - (target-system (assq-ref system 'target)) - (common-system (assq-ref system 'common)) - - (base-builder (assq-ref builder 'base)) - (target-builder (assq-ref builder 'target)) - (common-builder (assq-ref builder 'common)) - - (base-args (assq-ref args 'base)) - (target-args (assq-ref args 'target)) - (common-args (assq-ref args 'common))) - - (let ((matched-outputs (append-map get-derivation-data - (list (assq-ref outputs 'base) - (assq-ref outputs 'target) - (assq-ref outputs 'common)))) - (matched-inputs (append-map get-derivation-data - (list (assq-ref inputs 'base) - (assq-ref inputs 'target)))) - (matched-sources (append-map get-derivation-data - (list (assq-ref sources 'base) - (assq-ref sources 'target) - (assq-ref sources 'common))))) - (case (most-appropriate-mime-type - '(application/json text/html) - mime-types) - ((application/json) - (render-json - `((revision - . ((base - . ((derivation . ,base-derivation))) - (target - . ((derivation . ,target-derivation))))) - (outputs - . ,((lambda (l) (cond - ((= (length l) 3) `((base . ,(first l)) - (target . ,(second l)) - (common . ,(third l)))) - ((= (length l) 2) `((base . ,(first l)) - (target . ,(second l)))) - (else `((common . ,(first l)))))) - matched-outputs)) - (inputs - . ((base . ,(first matched-inputs)) - (target . ,(second matched-inputs)))) - (source - . ((base . ,(first matched-sources)) - (target . ,(second matched-sources)) - (common . ,(third matched-sources)))) - (system - . ((common . ,common-system))) - (builder-and-arguments - . ((builder . ,common-builder) - (arguments - . ((base . ,(list->vector - base-args)) - (target . ,(list->vector - target-args)))))) - (environment-variables . ,environment-variables)) - #:extra-headers http-headers-for-unchanging-content)) - (else - (render-html - #:sxml (compare/derivation - query-parameters - data) - #:extra-headers http-headers-for-unchanging-content)))))))))) + target-derivation)))) + (get-derivation-data + (lambda (items) + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t")))))) + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output))))) + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '()))))) + + (case (most-appropriate-mime-type + '(application/json text/html) + mime-types) + ((application/json) + (let* ((outputs (assq-ref data 'outputs)) + (base-outputs (append-map get-derivation-data (list (assq-ref outputs 'base)))) + (target-outputs (append-map get-derivation-data (list (assq-ref outputs 'target)))) + (common-outputs (append-map get-derivation-data (list (assq-ref outputs 'common)))) + + (inputs (assq-ref data 'inputs)) + (base-inputs (append-map get-derivation-data (list (assq-ref inputs 'base)))) + (target-inputs (append-map get-derivation-data (list (assq-ref inputs 'target)))) + + (sources (assq-ref data 'sources)) + (base-sources (append-map get-derivation-data (list (assq-ref sources 'base)))) + (target-sources (append-map get-derivation-data (list (assq-ref sources 'target)))) + (common-sources (append-map get-derivation-data (list (assq-ref sources 'common)))) + + (system (assq-ref data 'system)) + (base-system (assq-ref system 'base)) + (target-system (assq-ref system 'target)) + (common-system (assq-ref system 'common)) + + (builder (assq-ref data 'builder)) + (base-builder (assq-ref builder 'base)) + (target-builder (assq-ref builder 'target)) + (common-builder (assq-ref builder 'common)) + + (args (assq-ref data 'arguments)) + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target)) + (common-args (assq-ref args 'common)) + (environment-variables (assq-ref data 'environment-variables))) + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ((base . ,(list->vector base-outputs)) + (target . ,(list->vector target-outputs)) + (common . ,(list->vector common-outputs)))) + (inputs + . ((base . ,(list->vector base-inputs)) + (target . ,(list->vector target-inputs)))) + (sources + . ((base . ,(list->vector base-sources)) + (target . ,(list->vector target-sources)) + (common . ,(list->vector common-sources)))) + (system + . ((common . ,common-system))) + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,environment-variables)) + #:extra-headers http-headers-for-unchanging-content))) + (else + (render-html + #:sxml (compare/derivation + query-parameters + data) + #:extra-headers http-headers-for-unchanging-content))))))) (define (render-compare/package-derivations mime-types query-parameters) -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-15 16:09 ` Luciana Lima Brito @ 2021-04-15 23:19 ` Christopher Baines 2021-04-16 15:07 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-15 23:19 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 2370 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Thu, 15 Apr 2021 09:46:12 +0100 > Christopher Baines <mail@cbaines.net> wrote: > > Hi, > >> I'm not quite sure how to apply this, I'd suggest using git >> format-patch to generate the file next time as I think there would >> normally be some metadata along with the diff. > > I tried using git format-patch and I got 7 patches from my 7 commits, > then I generate a single patch output, which is attached. > The last commit before my modifications is this: > 410f58cb43f083623885a430700c6818a187cadc Ok, I looked at the overall diff, and it looks to me like these changes should probably be one commit. >> I would consider whether it's useful to have all these let blocks, and >> whether here is the right place for them. > >> Taking a binding like outputs, it's only used in a later let. You can >> do something like this (with let*) to remove the need to have >> multiple let blocks. > >> Also, since matched-outputs is only used when rendering the JSON, I'd >> move all the bindings that are only used for the JSON output within >> that part of the case statement, so that it's clearer that they only >> apply to that bit of the code. >> >> Does that make sense? > > I did it, I used the let* and this helped a lot. I also moved > everything into the case branch of the json. > >> I'm not sure what revision here referrs to. > > It was a placeholder, but now I removed it. > >> I hope that helps, just let me know if you have any questions, > > The function get-derivation-data does not depend on anything, don't you > think it goes better in another place outside render-compare/derivation? On the get-derivation-data function, I wouldn't use the same function to process the inputs, outputs and sources. The data for each is different, so I would separate the code as well. To avoid having to call a procedure three times, on the base, target and common items, I'd consider following the same pattern in the HTML generating code, map over a list of the lists, so something like: (map (lambda (name data) (cons name (match data ((name path hash-alg hash recursive) ...)))) '(base target common) (list (assq-ref outputs 'base) (assq-ref outputs 'target) (assq-ref outputs 'common))) Does that make sense? [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-15 23:19 ` Christopher Baines @ 2021-04-16 15:07 ` Luciana Lima Brito 2021-04-16 15:47 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-16 15:07 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel On Fri, 16 Apr 2021 00:19:46 +0100 Christopher Baines <mail@cbaines.net> wrote: Hi, > Ok, I looked at the overall diff, and it looks to me like these > changes should probably be one commit. I don't actually understand what you mean saying it should be one commit. Do I have to make my seven commits become a single one? How do I do that? > On the get-derivation-data function, I wouldn't use the same function > to process the inputs, outputs and sources. The data for each is > different, so I would separate the code as well. I understand that, but the logic to map the values for these three bindings is the same, wouldn't it generate redundancies implementing the same logic separately? > > To avoid having to call a procedure three times, on the base, target > and common items, I'd consider following the same pattern in the HTML > generating code, map over a list of the lists, so something like: > > (map (lambda (name data) > (cons name > (match data > ((name path hash-alg hash recursive) > ...)))) > '(base target common) > (list (assq-ref outputs 'base) > (assq-ref outputs 'target) > (assq-ref outputs 'common))) > > Does that make sense? Actually I did it in a similar way before, but it resulted in a list with all the values for base, target and common together, in which I had to have another way to separate them and render on json, for example, I tried appending "base", "target" or "common" names to each list (similar to your function?), but them I had to convert this list to a vector. Calling the function for each separately gave me a cleaner output. Also, I think that sometimes you might have more than one output for base, target like it does for common, and I fail to see how your example function addresses this. In short, I couldn't see the benefit of this over calling the function three times. Is it for organizational purpose or am I simply wrong? This time I'm just not understanding. :) -- Best Regards, Luciana Lima Brito MSc. in Computer Science ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-16 15:07 ` Luciana Lima Brito @ 2021-04-16 15:47 ` Christopher Baines 2021-04-16 18:46 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-16 15:47 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 3170 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Fri, 16 Apr 2021 00:19:46 +0100 > Christopher Baines <mail@cbaines.net> wrote: > > Hi, > >> Ok, I looked at the overall diff, and it looks to me like these >> changes should probably be one commit. > > I don't actually understand what you mean saying it should be one > commit. Do I have to make my seven commits become a single one? How do > I do that? From looking at the content of your commits, I think they should be merged together. There's some information about that here for example: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#_squashing >> On the get-derivation-data function, I wouldn't use the same function >> to process the inputs, outputs and sources. The data for each is >> different, so I would separate the code as well. > > I understand that, but the logic to map the values for these three > bindings is the same, wouldn't it generate redundancies implementing > the same logic separately? I'm unsure three bindings are you referring to, can you clairfy? >> To avoid having to call a procedure three times, on the base, target >> and common items, I'd consider following the same pattern in the HTML >> generating code, map over a list of the lists, so something like: >> >> (map (lambda (name data) >> (cons name >> (match data >> ((name path hash-alg hash recursive) >> ...)))) >> '(base target common) >> (list (assq-ref outputs 'base) >> (assq-ref outputs 'target) >> (assq-ref outputs 'common))) >> >> Does that make sense? > > Actually I did it in a similar way before, but it resulted in a list > with all the values for base, target and common together, in which > I had to have another way to separate them and render on json, for > example, I tried appending "base", "target" or "common" names to each > list (similar to your function?), but them I had to convert this list to > a vector. Getting a list with all of the values in individually was possibly due to using append-map rather than map. > Calling the function for each separately gave me a cleaner > output. Also, I think that sometimes you might have more than one > output for base, target like it does for common, and I fail to see how > your example function addresses this. In short, I couldn't see the > benefit of this over calling the function three times. Is it for > organizational purpose or am I simply wrong? This time I'm just not > understanding. It's an organisational thing, code is generally more readable if the scope for variables is tight and there's less indirection. Defining a procedure is one form of indirection. It's often really helpful, but I think it's unnecessary here. You're right though about the above example not handling data being a list, I think that's a fixable problem though, rather than the (match data ...) bit, I'd suggest using map with match-lambda, probably wrapped with list->vector if you want a vector which will be outputted as a JSON array. Does that make sense? [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-16 15:47 ` Christopher Baines @ 2021-04-16 18:46 ` Luciana Lima Brito 2021-04-16 19:17 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-16 18:46 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel On Fri, 16 Apr 2021 16:47:10 +0100 Christopher Baines <mail@cbaines.net> wrote: Hi > From looking at the content of your commits, I think they should be > merged together. > > There's some information about that here for example: > > https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#_squashing Thanks, I'll look into it. > >> On the get-derivation-data function, I wouldn't use the same > >> function to process the inputs, outputs and sources. The data for > >> each is different, so I would separate the code as well. > > > > I understand that, but the logic to map the values for these three > > bindings is the same, wouldn't it generate redundancies implementing > > the same logic separately? > > I'm unsure three bindings are you referring to, can you clairfy? The bindings I was talking about are "outputs", "inputs" and "sources", the only difference between them are the number of elements each one has, that is why I simply made a match clause for each one. If you think it is better I can separate them. Also, do you think it would be clearer if I move each one inside the json case branch? > >> To avoid having to call a procedure three times, on the base, > >> target and common items, I'd consider following the same pattern > >> in the HTML generating code, map over a list of the lists, so > >> something like: > >> > >> (map (lambda (name data) > >> (cons name > >> (match data > >> ((name path hash-alg hash recursive) > >> ...)))) > >> '(base target common) > >> (list (assq-ref outputs 'base) > >> (assq-ref outputs 'target) > >> (assq-ref outputs 'common))) > >> > >> Does that make sense? > > > > Actually I did it in a similar way before, but it resulted in a list > > with all the values for base, target and common together, in which > > I had to have another way to separate them and render on json, for > > example, I tried appending "base", "target" or "common" names to > > each list (similar to your function?), but them I had to convert > > this list to a vector. > > Getting a list with all of the values in individually was possibly due > to using append-map rather than map. > > > Calling the function for each separately gave me a cleaner > > output. Also, I think that sometimes you might have more than one > > output for base, target like it does for common, and I fail to see > > how your example function addresses this. In short, I couldn't see > > the benefit of this over calling the function three times. Is it for > > organizational purpose or am I simply wrong? This time I'm just not > > understanding. > > It's an organisational thing, code is generally more readable if the > scope for variables is tight and there's less indirection. Defining a > procedure is one form of indirection. It's often really helpful, but I > think it's unnecessary here. > > You're right though about the above example not handling data being a > list, I think that's a fixable problem though, rather than the (match > data ...) bit, I'd suggest using map with match-lambda, probably > wrapped with list->vector if you want a vector which will be > outputted as a JSON array. > > Does that make sense? I still don't quite get it. The function I had before was like this (using inputs as example): (append-map (lambda (label items) (map (match-lambda ((derivation output) (...)) (or items '())))) (list "base" "target" "common") (list (assq-ref inputs 'base) (assq-ref inputs 'target) (assq-ref inputs 'common))) Which indeed I made based on the html code. However this outputs me something like: (("base" derivation output) ("target" derivation output) ("common" derivation output) ...) where "..." are lots of different ("common" derivation output) lists. The only way I could think of showing this, was transforming it to a vector, which gives us the indexes from 0, and inside each one we would have the label showing where it came from. Is that the way you think it is better? (is this what your proposed function should accomplish?) I think the above output produces a bit less clean json, that is why I changed this function to the last one I sent you, so I don't need to pass any labels, because if I pass the base, target, and common lists separately the output is already correct and we don't need any vectors. -- Best Regards, Luciana Lima Brito MSc. in Computer Science ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-16 18:46 ` Luciana Lima Brito @ 2021-04-16 19:17 ` Christopher Baines 2021-04-16 22:47 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-16 19:17 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 6063 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Fri, 16 Apr 2021 16:47:10 +0100 > Christopher Baines <mail@cbaines.net> wrote: > >> >> On the get-derivation-data function, I wouldn't use the same >> >> function to process the inputs, outputs and sources. The data for >> >> each is different, so I would separate the code as well. >> > >> > I understand that, but the logic to map the values for these three >> > bindings is the same, wouldn't it generate redundancies implementing >> > the same logic separately? >> >> I'm unsure three bindings are you referring to, can you clairfy? > > The bindings I was talking about are "outputs", "inputs" and "sources", > the only difference between them are the number of elements each one > has, that is why I simply made a match clause for each one. If you think > it is better I can separate them. Also, do you think it would be clearer > if I move each one inside the json case branch? Ok, I think the data in the inputs, outputs and sources will be quite different, because they're different things, right? Looking at this page as an example [1], the inputs are each a derivation and a string saying which output of that derivation is being used as the input. The outputs have a number of bits of information, the name, path, hash algorithm, hash and whether it's recursive, and the sources are just a single string, the file. 1: https://data.guix.gnu.org/compare/derivation?base_derivation=/gnu/store/5s9mi0k3mk6cmakyckj2fcp4qb75rn79-debootstrap-1.0.123.drv&target_derivation=/gnu/store/vqhfs0dmfm4q0lr1w6sdhrbc2pa2jrar-debootstrap-1.0.123.drv If it's the same function being used to process the inputs, outputs and sources, then it's harder to quickly tell from the code what's going on, as you don't know which code inside get-derivation-data applies to the inputs, outputs or sources. Matching the code up directly with the data it should process will make this much more obvious. >> >> To avoid having to call a procedure three times, on the base, >> >> target and common items, I'd consider following the same pattern >> >> in the HTML generating code, map over a list of the lists, so >> >> something like: >> >> >> >> (map (lambda (name data) >> >> (cons name >> >> (match data >> >> ((name path hash-alg hash recursive) >> >> ...)))) >> >> '(base target common) >> >> (list (assq-ref outputs 'base) >> >> (assq-ref outputs 'target) >> >> (assq-ref outputs 'common))) >> >> >> >> Does that make sense? >> > >> > Actually I did it in a similar way before, but it resulted in a list >> > with all the values for base, target and common together, in which >> > I had to have another way to separate them and render on json, for >> > example, I tried appending "base", "target" or "common" names to >> > each list (similar to your function?), but them I had to convert >> > this list to a vector. >> >> Getting a list with all of the values in individually was possibly due >> to using append-map rather than map. >> >> > Calling the function for each separately gave me a cleaner >> > output. Also, I think that sometimes you might have more than one >> > output for base, target like it does for common, and I fail to see >> > how your example function addresses this. In short, I couldn't see >> > the benefit of this over calling the function three times. Is it for >> > organizational purpose or am I simply wrong? This time I'm just not >> > understanding. >> >> It's an organisational thing, code is generally more readable if the >> scope for variables is tight and there's less indirection. Defining a >> procedure is one form of indirection. It's often really helpful, but I >> think it's unnecessary here. >> >> You're right though about the above example not handling data being a >> list, I think that's a fixable problem though, rather than the (match >> data ...) bit, I'd suggest using map with match-lambda, probably >> wrapped with list->vector if you want a vector which will be >> outputted as a JSON array. >> >> Does that make sense? > > I still don't quite get it. The function I had before was like this > (using inputs as example): > > (append-map > (lambda (label items) > (map > (match-lambda > ((derivation output) > (...)) > (or items '())))) > (list "base" "target" "common") > (list (assq-ref inputs 'base) > (assq-ref inputs 'target) > (assq-ref inputs 'common))) > > Which indeed I made based on the html code. However this outputs me > something like: > > (("base" derivation output) > ("target" derivation output) > ("common" derivation output) > ...) > > where "..." are lots of different ("common" derivation output) lists. > > The only way I could think of showing this, was transforming it to a > vector, which gives us the indexes from 0, and inside each one we > would have the label showing where it came from. Is that the way you > think it is better? (is this what your proposed function should > accomplish?) > > I think the above output produces a bit less clean json, > that is why I changed this function to the last one I sent you, so I > don't need to pass any labels, because if I pass the base, target, and > common lists separately the output is already correct and we don't need > any vectors. So, append-map expects the procedure it runs on the elements in the lists its provided to return a list. Whereas map would give you a list of lists in this case, append-map effectively runs append on that list of lists, so you end up with a list. While a flatter list is what you want when building an HTML table, I think you were looking to get a JSON object separating the common, base and target elements, right? If so, then map, rather than append-map should be more useful to you here. Since above you're passing in two lists of three things, if the procedure passed to map returns a pair with a string in the first position, you'll end up producing the scheme version of a JSON object (an alist). [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-16 19:17 ` Christopher Baines @ 2021-04-16 22:47 ` Luciana Lima Brito 2021-04-17 8:40 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-16 22:47 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1024 bytes --] On Fri, 16 Apr 2021 20:17:45 +0100 Christopher Baines <mail@cbaines.net> wrote: Hi, I hope the patch is correct this time. I considered all you said, so I separated the functions to get outputs, inputs and sources. I also implemented everything inside the case of the json/application. > While a flatter list is what you want when building an HTML table, I > think you were looking to get a JSON object separating the common, > base and target elements, right? If so, then map, rather than > append-map should be more useful to you here. Since above you're > passing in two lists of three things, if the procedure passed to map > returns a pair with a string in the first position, you'll end up > producing the scheme version of a JSON object (an alist). You were right about that, I'm using map now. Please, let me know if I missed something. Thanks in advance, I'm learning a great deal! :) -- Best Regards, Luciana Lima Brito MSc. in Computer Science Federal University of Uberlândia [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 7404 bytes --] From b51dd007180e69f7da479a857afd48dfa60e32e7 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 122 ++++++++++++++++++- 1 file changed, 119 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..1b0fc2f 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,125 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs (assq-ref data 'outputs)) + (matched-outputs + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t"))))))) + (or items '())))) + (list "base" "target" "common") + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + (base-outputs (list->vector (cdr (first matched-outputs)))) + (target-outputs (list->vector (cdr (second matched-outputs)))) + (common-outputs (list->vector (cdr (third matched-outputs)))) + + (inputs (assq-ref data 'inputs)) + (matched-inputs + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output)))))) + (or items '())))) + (list "base" "target" "common") + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + (base-inputs (list->vector (cdr (first matched-inputs)))) + (target-inputs (list->vector (cdr (second matched-inputs)))) + + (sources (assq-ref data 'sources)) + (matched-sources + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '())))) + (list "base" "target" "common") + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + (base-sources (list->vector (cdr (first matched-sources)))) + (target-sources (list->vector (cdr (second matched-sources)))) + (common-sources (list->vector (cdr (third matched-sources)))) + + (system (assq-ref data 'system)) + (base-system (assq-ref system 'base)) + (target-system (assq-ref system 'target)) + (common-system (assq-ref system 'common)) + + (builder (assq-ref data 'builder)) + (base-builder (assq-ref builder 'base)) + (target-builder (assq-ref builder 'target)) + (common-builder (assq-ref builder 'common)) + + (args (assq-ref data 'arguments)) + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target)) + (common-args (assq-ref args 'common)) + (environment-variables (assq-ref data 'environment-variables))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ((,(first (first matched-outputs)) . ,base-outputs) + (,(first (second matched-outputs)) . ,target-outputs) + (,(first (third matched-outputs)) . ,common-outputs))) + (inputs + . ((,(first (first matched-inputs)) . ,base-inputs) + (,(first (second matched-inputs)) . ,target-inputs))) + (sources + . ((,(first (first matched-sources)) . ,base-sources) + (,(first (second matched-sources)) . ,target-sources) + (,(first (third matched-sources)) . ,common-sources))) + (system + . ((common . ,common-system))) + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,environment-variables)) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-16 22:47 ` Luciana Lima Brito @ 2021-04-17 8:40 ` Christopher Baines 2021-04-17 12:48 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-17 8:40 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1428 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Fri, 16 Apr 2021 20:17:45 +0100 > Christopher Baines <mail@cbaines.net> wrote: > > Hi, > > I hope the patch is correct this time. > I considered all you said, so I separated the > functions to get outputs, inputs and sources. I also implemented > everything inside the case of the json/application. Yep, that's looking good, much neater. >> While a flatter list is what you want when building an HTML table, I >> think you were looking to get a JSON object separating the common, >> base and target elements, right? If so, then map, rather than >> append-map should be more useful to you here. Since above you're >> passing in two lists of three things, if the procedure passed to map >> returns a pair with a string in the first position, you'll end up >> producing the scheme version of a JSON object (an alist). > > You were right about that, I'm using map now. > > Please, let me know if I missed something. > Thanks in advance, I'm learning a great deal! :) I think you're getting there, but it looks like you're close to what you want with matched-outputs say, and then later you pick bits out of that alist, generate vectors from the lists, and then rebuild the alist. I think you can remove all that complexity by just tweaking what you're doing up when you generate matched-outputs. I think this is true for matched-outputs, matched-inputs and matched-sources. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-17 8:40 ` Christopher Baines @ 2021-04-17 12:48 ` Luciana Lima Brito 2021-04-17 13:11 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-17 12:48 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 773 bytes --] On Sat, 17 Apr 2021 09:40:22 +0100 Christopher Baines <mail@cbaines.net> wrote: > I think you're getting there, but it looks like you're close to what > you want with matched-outputs say, and then later you pick bits out > of that alist, generate vectors from the lists, and then rebuild the > alist. I think you can remove all that complexity by just tweaking > what you're doing up when you generate matched-outputs. I think this > is true for matched-outputs, matched-inputs and matched-sources. It's beautiful! Now I could understand better the alist, and things make much more sense to me. I simplified this part of getting properly the values from matched-outputs, matched-inputs and matched-sources. -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 6782 bytes --] From 8d91269ead953c7d087242fbce5857af89af3025 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 114 ++++++++++++++++++- 1 file changed, 111 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..d05c177 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,117 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs (assq-ref data 'outputs)) + (matched-outputs + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t"))))))) + (or items '())))) + '(base target common) + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + + (inputs (assq-ref data 'inputs)) + (matched-inputs + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output)))))) + (or items '())))) + '(base target common) + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + + (sources (assq-ref data 'sources)) + (matched-sources + (map + (lambda (label items) + (cons label + (map + (match-lambda + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '())))) + '(base target common) + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + + (system (assq-ref data 'system)) + (base-system (assq-ref system 'base)) + (target-system (assq-ref system 'target)) + (common-system (assq-ref system 'common)) + + (builder (assq-ref data 'builder)) + (base-builder (assq-ref builder 'base)) + (target-builder (assq-ref builder 'target)) + (common-builder (assq-ref builder 'common)) + + (args (assq-ref data 'arguments)) + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target)) + (common-args (assq-ref args 'common)) + (environment-variables (assq-ref data 'environment-variables))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ((base . ,(list->vector (assq-ref matched-outputs 'base))) + (target . ,(list->vector (assq-ref matched-outputs 'target))) + (common . ,(list->vector (assq-ref matched-outputs 'common))))) + (inputs + . ((base . ,(list->vector (assq-ref matched-inputs 'base))) + (target . ,(list->vector (assq-ref matched-inputs 'target))))) + (sources + . ((base . ,(list->vector (assq-ref matched-sources 'base))) + (target . ,(list->vector (assq-ref matched-sources 'target))) + (common . ,(list->vector (assq-ref matched-sources 'common))))) + (system + . ((common . ,common-system))) + (builder-and-arguments + . ((builder . ,common-builder) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,environment-variables)) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-17 12:48 ` Luciana Lima Brito @ 2021-04-17 13:11 ` Christopher Baines 2021-04-17 14:08 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-17 13:11 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1279 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Sat, 17 Apr 2021 09:40:22 +0100 > Christopher Baines <mail@cbaines.net> wrote: > >> I think you're getting there, but it looks like you're close to what >> you want with matched-outputs say, and then later you pick bits out >> of that alist, generate vectors from the lists, and then rebuild the >> alist. I think you can remove all that complexity by just tweaking >> what you're doing up when you generate matched-outputs. I think this >> is true for matched-outputs, matched-inputs and matched-sources. > > It's beautiful! > Now I could understand better the alist, and things make much more > sense to me. I simplified this part of getting properly the values from > matched-outputs, matched-inputs and matched-sources. I think that's better, but you're still taking the matched-outputs, matched-inputs and matched-sources alists, then later making the values vectors rather than lists. The code would be even simpler if you converted to a vector just after the map that produces the relevant lists. I haven't looked to closely at the rest, but it looks like the environment-variables binding is unnecessary, given how simple the bit of code used is, it can just be inlined in the one place where the binding is used. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-17 13:11 ` Christopher Baines @ 2021-04-17 14:08 ` Luciana Lima Brito 2021-04-17 17:45 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-17 14:08 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 739 bytes --] On Sat, 17 Apr 2021 14:11:37 +0100 Christopher Baines <mail@cbaines.net> wrote: Hi, > I think that's better, but you're still taking the matched-outputs, > matched-inputs and matched-sources alists, then later making the > values vectors rather than lists. The code would be even simpler if > you converted to a vector just after the map that produces the > relevant lists. > > I haven't looked to closely at the rest, but it looks like the > environment-variables binding is unnecessary, given how simple the bit > of code used is, it can just be inlined in the one place where the > binding is used. Done! :) I even simplified some other stuff, based on what you said. -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 5754 bytes --] From 701c89e8f039a6bc7d9b616acded54eac26fb0a7 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 100 ++++++++++++++++++- 1 file changed, 97 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..14a25ee 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,103 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs (assq-ref data 'outputs)) + (matched-outputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `(,@(if (null? name) + '() + `((name . ,name))) + ,@(if (null? path) + '() + `((path . ,path)) + ) + ,@(if (or (null? hash-alg) (not (string? hash-alg))) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (or (null? hash) (not (string? hash))) + '() + `((hash . ,hash)) + ) + ,@(if (null? recursive) + '() + `((recursive . ,(string=? recursive "t"))))))) + (or items '()))))) + '(base target common) + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + + (inputs (assq-ref data 'inputs)) + (matched-inputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation output) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation))) + ,@(if (null? output) + '() + `((output . ,output)))))) + (or items '()))))) + '(base target common) + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + + (sources (assq-ref data 'sources)) + (matched-sources + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation) + `(,@(if (null? derivation) + '() + `((derivation . ,derivation)))))) + (or items '()))))) + '(base target common) + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + + (args (assq-ref data 'arguments)) + (base-args (assq-ref args 'base)) + (target-args (assq-ref args 'target))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ,matched-outputs) + (inputs + . ,matched-inputs) + (sources + . ,matched-sources) + (system + . ,(assq-ref data 'system)) + (builder-and-arguments + . ((builder . ,(assq-ref data 'builder)) + (arguments + . ((base . ,(list->vector + base-args)) + (target . ,(list->vector + target-args)))))) + (environment-variables . ,(assq-ref data 'environment-variables))) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-17 14:08 ` Luciana Lima Brito @ 2021-04-17 17:45 ` Christopher Baines 2021-04-18 13:12 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-17 17:45 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1285 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Sat, 17 Apr 2021 14:11:37 +0100 > Christopher Baines <mail@cbaines.net> wrote: > > Hi, > >> I think that's better, but you're still taking the matched-outputs, >> matched-inputs and matched-sources alists, then later making the >> values vectors rather than lists. The code would be even simpler if >> you converted to a vector just after the map that produces the >> relevant lists. >> >> I haven't looked to closely at the rest, but it looks like the >> environment-variables binding is unnecessary, given how simple the bit >> of code used is, it can just be inlined in the one place where the >> binding is used. > > Done! :) > I even simplified some other stuff, based on what you said. Great, I think that looks much simpler. Some more things to think about: - Variable naming, what does the "matched" in matched outputs mean? (same goes for the other "matched" things) - (if (null? ...), I'm unsure if all of those checks are necessary, I believe some fields at least will never be "null?". - Builder and arguments grouping, I think this makes sense on the HTML page, as they're connected, but does it make sense in the JSON? I think you're getting close to something that's ready to merge though. Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-17 17:45 ` Christopher Baines @ 2021-04-18 13:12 ` Luciana Lima Brito 2021-04-18 13:19 ` Luciana Lima Brito 2021-04-18 16:34 ` Christopher Baines 0 siblings, 2 replies; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-18 13:12 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1193 bytes --] Hi, On Sat, 17 Apr 2021 18:45:14 +0100 Christopher Baines <mail@cbaines.net> wrote: > Some more things to think about: > > - Variable naming, what does the "matched" in matched outputs mean? > (same goes for the other "matched" things) The name matched would refer to the match function, but I changed to *-values. The names I wanted were "outputs", "inputs" and "sources", but I already used them. If you have anything in mind, please let me know. > - (if (null? ...), I'm unsure if all of those checks are necessary, I > believe some fields at least will never be "null?". I revised it, I think now it's better. About the "recursive" field, apparently it assumes a string value "t" or "f", and I convert this to a boolean. Are there other values possible? > - Builder and arguments grouping, I think this makes sense on the > HTML page, as they're connected, but does it make sense in the JSON? indeed, I separated them. > I think you're getting close to something that's ready to merge > though. One last thing, I see that on the html the commom inputs are ommited. Does this make sense for the json too? -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 4993 bytes --] From 614ea09fc6a2d96a9dc2955ae08736740d08f8f8 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 89 +++++++++++++++++++- 1 file changed, 86 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..09caa82 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,92 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs (assq-ref data 'outputs)) + (output-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `((name . ,name) + (path . ,path) + ,@(if (not (string? hash-alg)) + '() + `((hash-algorithm . ,hash-alg)) + ) + ,@(if (not (string? hash)) + '() + `((hash . ,hash)) + ) + (recursive . ,(string=? recursive "t")) + ) + )) + (or items '()))))) + '(base target common) + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + + (inputs (assq-ref data 'inputs)) + (input-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation output) + `((derivation . ,derivation) + (output . ,output) + ))) + (or items '()))))) + '(base target common) + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + + (sources (assq-ref data 'sources)) + (source-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation) + `((derivation . ,derivation) + ))) + (or items '()))))) + '(base target common) + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + + (args (assq-ref data 'arguments)) + (base-args (list->vector (assq-ref args 'base))) + (target-args (list->vector (assq-ref args 'target)))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ,output-values) + (inputs + . ,input-values) + (sources + . ,source-values) + (system + . ,(assq-ref data 'system)) + (builder + . ,(assq-ref data 'builder)) + (arguments + . ((base . ,base-args) + (target . ,target-args))) + (environment-variables . ,(assq-ref data 'environment-variables))) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-18 13:12 ` Luciana Lima Brito @ 2021-04-18 13:19 ` Luciana Lima Brito 2021-04-18 16:34 ` Christopher Baines 1 sibling, 0 replies; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-18 13:19 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1438 bytes --] On Sun, 18 Apr 2021 13:12:07 +0000 Luciana Lima Brito <lubrito@posteo.net> wrote: > Hi, > > On Sat, 17 Apr 2021 18:45:14 +0100 > Christopher Baines <mail@cbaines.net> wrote: > > > Some more things to think about: > > > > - Variable naming, what does the "matched" in matched outputs mean? > > (same goes for the other "matched" things) > > The name matched would refer to the match function, but I changed to > *-values. The names I wanted were "outputs", "inputs" > and "sources", but I already used them. If you have anything in mind, > please let me know. > > > - (if (null? ...), I'm unsure if all of those checks are > > necessary, I believe some fields at least will never be "null?". > > I revised it, I think now it's better. > About the "recursive" field, apparently it assumes a string value "t" > or "f", and I convert this to a boolean. Are there other values > possible? > > > - Builder and arguments grouping, I think this makes sense on the > > HTML page, as they're connected, but does it make sense in the > > JSON? > > indeed, I separated them. > > > I think you're getting close to something that's ready to merge > > though. > > > One last thing, I see that on the html the commom inputs are ommited. > Does this make sense for the json too? > The last patch had a few misaligned parens, please disregard. This one is fixed. -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 4741 bytes --] From ff348cb5ce7db9ce9f08a6f0827356faa6465877 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 83 +++++++++++++++++++- 1 file changed, 80 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..2ff7a40 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,86 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs (assq-ref data 'outputs)) + (output-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `((name . ,name) + (path . ,path) + ,@(if (not (string? hash-alg)) + '() + `((hash-algorithm . ,hash-alg))) + ,@(if (not (string? hash)) + '() + `((hash . ,hash))) + (recursive . ,(string=? recursive "t"))))) + (or items '()))))) + '(base target common) + (list (assq-ref outputs 'base) + (assq-ref outputs 'target) + (assq-ref outputs 'common)))) + + (inputs (assq-ref data 'inputs)) + (input-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation output) + `((derivation . ,derivation) + (output . ,output)))) + (or items '()))))) + '(base target common) + (list (assq-ref inputs 'base) + (assq-ref inputs 'target)))) + + (sources (assq-ref data 'sources)) + (source-values + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation) + `((derivation . ,derivation)))) + (or items '()))))) + '(base target common) + (list (assq-ref sources 'base) + (assq-ref sources 'target) + (assq-ref sources 'common)))) + + (args (assq-ref data 'arguments)) + (base-args (list->vector (assq-ref args 'base))) + (target-args (list->vector (assq-ref args 'target)))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ,output-values) + (inputs + . ,input-values) + (sources + . ,source-values) + (system + . ,(assq-ref data 'system)) + (builder + . ,(assq-ref data 'builder)) + (arguments + . ((base . ,base-args) + (target . ,target-args))) + (environment-variables . ,(assq-ref data 'environment-variables))) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-18 13:12 ` Luciana Lima Brito 2021-04-18 13:19 ` Luciana Lima Brito @ 2021-04-18 16:34 ` Christopher Baines 2021-04-18 19:12 ` Luciana Lima Brito 1 sibling, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-18 16:34 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1933 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > Hi, > > On Sat, 17 Apr 2021 18:45:14 +0100 > Christopher Baines <mail@cbaines.net> wrote: > >> Some more things to think about: >> >> - Variable naming, what does the "matched" in matched outputs mean? >> (same goes for the other "matched" things) > > The name matched would refer to the match function, but I changed to > *-values. The names I wanted were "outputs", "inputs" > and "sources", but I already used them. If you have anything in mind, > please let me know. I think it might be good to do something, just to narrow the scope. The outputs binding is valid for the whole let*, and all the code in it, but is only used on three lines, in one single place. Maybe there could be a let there that just defines outputs (maybe named output-groups so you can use the outputs binding for the overall thing). >> - (if (null? ...), I'm unsure if all of those checks are necessary, I >> believe some fields at least will never be "null?". > > I revised it, I think now it's better. > About the "recursive" field, apparently it assumes a string value "t" > or "f", and I convert this to a boolean. Are there other values > possible? That's a good question, I'd look at the database schema, assuming the type of the field is a boolean, the question is whether the field is nullable? >> I think you're getting close to something that's ready to merge >> though. > > One last thing, I see that on the html the commom inputs are ommited. > Does this make sense for the json too? Hmm, I'm not sure why that is on the HTML page, but I'd generally try and keep most bits in the JSON, since it's not as helpful to omit bits if they're not that important. One other thing I noticed is that the alist for the args is being picked apart then reconstructed. Like for the inputs, outputs and sources, I'd map over the arguments alist and transform it to the way you want it to be. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-18 16:34 ` Christopher Baines @ 2021-04-18 19:12 ` Luciana Lima Brito 2021-04-19 8:26 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-18 19:12 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1464 bytes --] Hi, On Sun, 18 Apr 2021 17:34:13 +0100 Christopher Baines <mail@cbaines.net> wrote: > I think it might be good to do something, just to narrow the scope. > The outputs binding is valid for the whole let*, and all the code in > it, but is only used on three lines, in one single place. Maybe there > could be a let there that just defines outputs (maybe named > output-groups so you can use the outputs binding for the overall > thing). I did it. > That's a good question, I'd look at the database schema, assuming the > type of the field is a boolean, the question is whether the field is > nullable? I looked on the database schema, and the "recursive" field is not nullable, and it is a boolean, so the test I'm doing is working for this. > Hmm, I'm not sure why that is on the HTML page, but I'd generally try > and keep most bits in the JSON, since it's not as helpful to omit bits > if they're not that important. I added the "common" field for inputs. > > One other thing I noticed is that the alist for the args is being > picked apart then reconstructed. Like for the inputs, outputs and > sources, I'd map over the arguments alist and transform it to the way > you want it to be. This part was a bit more complicated for me to understand. You mean I should build a function similar to outputs, inputs and sources to map the arguments, wouldn't it be a lot just to show a vector? -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 4839 bytes --] From dc74d1a8f8f5e7527cdb63b66e8e2b937e614f32 Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 83 +++++++++++++++++++- 1 file changed, 80 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..ab5f9c4 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,86 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let* ((outputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `((name . ,name) + (path . ,path) + ,@(if (not (string? hash-alg)) + '() + `((hash-algorithm . ,hash-alg))) + ,@(if (not (string? hash)) + '() + `((hash . ,hash))) + (recursive . ,(string=? recursive "t"))))) + (or items '()))))) + '(base target common) + (let ((output-groups (assq-ref data 'outputs))) + (list (assq-ref output-groups 'base) + (assq-ref output-groups 'target) + (assq-ref output-groups 'common))))) + + (inputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation output) + `((derivation . ,derivation) + (output . ,output)))) + (or items '()))))) + '(base target common) + (let ((input-groups (assq-ref data 'inputs))) + (list (assq-ref input-groups 'base) + (assq-ref input-groups 'target) + (assq-ref input-groups 'common))))) + + (sources + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation) + `((derivation . ,derivation)))) + (or items '()))))) + '(base target common) + (let ((source-groups (assq-ref data 'sources))) + (list (assq-ref source-groups 'base) + (assq-ref source-groups 'target) + (assq-ref source-groups 'common))))) + + (args (assq-ref data 'arguments)) + (base-args (list->vector (assq-ref args 'base))) + (target-args (list->vector (assq-ref args 'target)))) + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ,outputs) + (inputs + . ,inputs) + (sources + . ,sources) + (system + . ,(assq-ref data 'system)) + (builder + . ,(assq-ref data 'builder)) + (arguments + . ((base . ,base-args) + (target . ,target-args))) + (environment-variables . ,(assq-ref data 'environment-variables))) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-18 19:12 ` Luciana Lima Brito @ 2021-04-19 8:26 ` Christopher Baines 2021-04-19 14:04 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-19 8:26 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 2125 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > Hi, > > On Sun, 18 Apr 2021 17:34:13 +0100 > Christopher Baines <mail@cbaines.net> wrote: > >> I think it might be good to do something, just to narrow the scope. >> The outputs binding is valid for the whole let*, and all the code in >> it, but is only used on three lines, in one single place. Maybe there >> could be a let there that just defines outputs (maybe named >> output-groups so you can use the outputs binding for the overall >> thing). > > I did it. Great :) >> That's a good question, I'd look at the database schema, assuming the >> type of the field is a boolean, the question is whether the field is >> nullable? > > I looked on the database schema, and the "recursive" field is not > nullable, and it is a boolean, so the test I'm doing is working for > this. Cool. >> Hmm, I'm not sure why that is on the HTML page, but I'd generally try >> and keep most bits in the JSON, since it's not as helpful to omit bits >> if they're not that important. > > I added the "common" field for inputs. Ok. >> One other thing I noticed is that the alist for the args is being >> picked apart then reconstructed. Like for the inputs, outputs and >> sources, I'd map over the arguments alist and transform it to the way >> you want it to be. > > This part was a bit more complicated for me to understand. You mean I > should build a function similar to outputs, inputs and sources to map > the arguments, wouldn't it be a lot just to show a vector? As an example, if you have this structure. '((foo . 1) (bar . 2)) and you want the numbers to be strings, you can do something like: (map (match-lambda ((name . number) (cons name (number->string number)))) data) and I think a similar approach for the transformation you're trying to perform to the arguments will more cleanly represent what you're trying to do. One different thing I noticed: ,@(if (not (string? hash-alg)) '() `((hash-algorithm . ,hash-alg))) I'd suggest simplifying this by flipping the different parts of the if, and removing the not. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-19 8:26 ` Christopher Baines @ 2021-04-19 14:04 ` Luciana Lima Brito 2021-04-19 20:20 ` Christopher Baines 0 siblings, 1 reply; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-19 14:04 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 829 bytes --] On Mon, 19 Apr 2021 09:26:13 +0100 Christopher Baines <mail@cbaines.net> wrote: > As an example, if you have this structure. > > '((foo . 1) > (bar . 2)) > > and you want the numbers to be strings, you can do something like: > > (map (match-lambda > ((name . number) > (cons name (number->string number)))) > data) > > and I think a similar approach for the transformation you're trying to > perform to the arguments will more cleanly represent what you're > trying to do. It helped a lot! > One different thing I noticed: > > ,@(if (not (string? hash-alg)) > '() > `((hash-algorithm . ,hash-alg))) > > I'd suggest simplifying this by flipping the different parts of the > if, and removing the not. Done. -- Best Regards, Luciana Lima Brito MSc. in Computer Science [-- Attachment #2: 0001-Implement-basic-json-output-for-the-derivation-compa.patch --] [-- Type: text/x-patch, Size: 4739 bytes --] From ce5d97dd490f6ffb58548e83be78af576404b01f Mon Sep 17 00:00:00 2001 From: Luciana Brito <lubrito@posteo.net> Date: Sun, 11 Apr 2021 11:06:06 -0300 Subject: [PATCH] Implement basic json output for the derivation comparison page --- guix-data-service/web/compare/controller.scm | 85 +++++++++++++++++++- 1 file changed, 82 insertions(+), 3 deletions(-) diff --git a/guix-data-service/web/compare/controller.scm b/guix-data-service/web/compare/controller.scm index a6aa198..85fcbe9 100644 --- a/guix-data-service/web/compare/controller.scm +++ b/guix-data-service/web/compare/controller.scm @@ -588,9 +588,88 @@ '(application/json text/html) mime-types) ((application/json) - (render-json - '((error . "unimplemented")) ; TODO - #:extra-headers http-headers-for-unchanging-content)) + (let ((outputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((name path hash-alg hash recursive) + `((name . ,name) + (path . ,path) + ,@(if (string? hash-alg) + `((hash-algorithm . ,hash-alg)) + '()) + ,@(if (string? hash) + `((hash . ,hash)) + '()) + (recursive . ,(string=? recursive "t"))))) + (or items '()))))) + '(base target common) + (let ((output-groups (assq-ref data 'outputs))) + (list (assq-ref output-groups 'base) + (assq-ref output-groups 'target) + (assq-ref output-groups 'common))))) + + (inputs + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation output) + `((derivation . ,derivation) + (output . ,output)))) + (or items '()))))) + '(base target common) + (let ((input-groups (assq-ref data 'inputs))) + (list (assq-ref input-groups 'base) + (assq-ref input-groups 'target) + (assq-ref input-groups 'common))))) + + (sources + (map + (lambda (label items) + (cons label + (list->vector + (map + (match-lambda + ((derivation) + `((derivation . ,derivation)))) + (or items '()))))) + '(base target common) + (let ((source-groups (assq-ref data 'sources))) + (list (assq-ref source-groups 'base) + (assq-ref source-groups 'target) + (assq-ref source-groups 'common))))) + + (arguments + (map + (match-lambda + ((label args ...) + `(,label . ,(list->vector args)))) + (assq-ref data 'arguments)))) + + (render-json + `((base + . ((derivation . ,base-derivation))) + (target + . ((derivation . ,target-derivation))) + (outputs + . ,outputs) + (inputs + . ,inputs) + (sources + . ,sources) + (system + . ,(assq-ref data 'system)) + (builder + . ,(assq-ref data 'builder)) + (arguments . ,arguments) + (environment-variables . ,(assq-ref data 'environment-variables))) + #:extra-headers http-headers-for-unchanging-content))) (else (render-html #:sxml (compare/derivation -- 2.30.2 ^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-19 14:04 ` Luciana Lima Brito @ 2021-04-19 20:20 ` Christopher Baines 2021-04-19 20:56 ` Luciana Lima Brito 0 siblings, 1 reply; 22+ messages in thread From: Christopher Baines @ 2021-04-19 20:20 UTC (permalink / raw) To: Luciana Lima Brito; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 3653 bytes --] Luciana Lima Brito <lubrito@posteo.net> writes: > On Mon, 19 Apr 2021 09:26:13 +0100 > Christopher Baines <mail@cbaines.net> wrote: > >> As an example, if you have this structure. >> >> '((foo . 1) >> (bar . 2)) >> >> and you want the numbers to be strings, you can do something like: >> >> (map (match-lambda >> ((name . number) >> (cons name (number->string number)))) >> data) >> >> and I think a similar approach for the transformation you're trying to >> perform to the arguments will more cleanly represent what you're >> trying to do. > > It helped a lot! > >> One different thing I noticed: >> >> ,@(if (not (string? hash-alg)) >> '() >> `((hash-algorithm . ,hash-alg))) >> >> I'd suggest simplifying this by flipping the different parts of the >> if, and removing the not. > > Done. Great, I've gone ahead and pushed this now. I tweaked the following things, it's mostly removing trailing whitespace, and I changed the way the alist was formatted, just so it's a bit more compact (how it was before is OK as well). @@ -617,7 +617,7 @@ (lambda (label items) (cons label (list->vector - (map + (map (match-lambda ((derivation output) `((derivation . ,derivation) @@ -651,24 +651,18 @@ ((label args ...) `(,label . ,(list->vector args)))) (assq-ref data 'arguments)))) - + (render-json - `((base - . ((derivation . ,base-derivation))) - (target - . ((derivation . ,target-derivation))) - (outputs - . ,outputs) - (inputs - . ,inputs) - (sources - . ,sources) - (system - . ,(assq-ref data 'system)) - (builder - . ,(assq-ref data 'builder)) - (arguments . ,arguments) - (environment-variables . ,(assq-ref data 'environment-variables))) + `((base . ((derivation . ,base-derivation))) + (target . ((derivation . ,target-derivation))) + (outputs . ,outputs) + (inputs . ,inputs) + (sources . ,sources) + (system . ,(assq-ref data 'system)) + (builder . ,(assq-ref data 'builder)) + (arguments . ,arguments) + (environment-variables . ,(assq-ref + data 'environment-variables))) #:extra-headers http-headers-for-unchanging-content))) (else (render-html In terms of what to do next, you could continue on this derivation comparison path. Some of the code you've got here could be used to make the data better right when the database is queried. Take the recursive field for outputs for example, it would be better to convert it to a boolean where the database query is made. Also, there's some admin to do in terms of recording a contribution, and starting to think about submitting the final application. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 987 bytes --] ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Outreachy - Guix Data Service: implementing basic json output for derivation comparison page 2021-04-19 20:20 ` Christopher Baines @ 2021-04-19 20:56 ` Luciana Lima Brito 0 siblings, 0 replies; 22+ messages in thread From: Luciana Lima Brito @ 2021-04-19 20:56 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel Hi On Mon, 19 Apr 2021 21:20:18 +0100 Christopher Baines <mail@cbaines.net> wrote: > Great, I've gone ahead and pushed this now. What a great news!!! > In terms of what to do next, you could continue on this derivation > comparison path. Some of the code you've got here could be used to > make the data better right when the database is queried. Take the > recursive field for outputs for example, it would be better to > convert it to a boolean where the database query is made. Yes, I could do that. I will look into it. > Also, there's some admin to do in terms of recording a contribution, > and starting to think about submitting the final application. I think I have to do something about that too. -- Best Regards, Luciana Lima Brito MSc. in Computer Science ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2021-04-19 20:57 UTC | newest] Thread overview: 22+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2021-04-14 19:48 Outreachy - Guix Data Service: implementing basic json output for derivation comparison page Luciana Lima Brito 2021-04-15 8:46 ` Christopher Baines 2021-04-15 16:09 ` Luciana Lima Brito 2021-04-15 23:19 ` Christopher Baines 2021-04-16 15:07 ` Luciana Lima Brito 2021-04-16 15:47 ` Christopher Baines 2021-04-16 18:46 ` Luciana Lima Brito 2021-04-16 19:17 ` Christopher Baines 2021-04-16 22:47 ` Luciana Lima Brito 2021-04-17 8:40 ` Christopher Baines 2021-04-17 12:48 ` Luciana Lima Brito 2021-04-17 13:11 ` Christopher Baines 2021-04-17 14:08 ` Luciana Lima Brito 2021-04-17 17:45 ` Christopher Baines 2021-04-18 13:12 ` Luciana Lima Brito 2021-04-18 13:19 ` Luciana Lima Brito 2021-04-18 16:34 ` Christopher Baines 2021-04-18 19:12 ` Luciana Lima Brito 2021-04-19 8:26 ` Christopher Baines 2021-04-19 14:04 ` Luciana Lima Brito 2021-04-19 20:20 ` Christopher Baines 2021-04-19 20:56 ` Luciana Lima Brito
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.