[bug#35935] [PATCH] guix: import: simplify recursive import

[bug#35935] [PATCH] guix: import: simplify recursive import

[Robert Vollmert]

This simplifies the logic of recursive-import, intending no
major functional changes. The package import function is no
longer called twice per package. Failed imports now make it
to the package stream as '() instead of #f.

* guix/import/utils.scm: Simplify recursive-import.
---
 guix/import/utils.scm | 86 ++++++++++++++++---------------------------
 1 file changed, 32 insertions(+), 54 deletions(-)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 516c0cfaa2..ff548b809a 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -378,57 +378,35 @@ separated by PRED."
                            #:allow-other-keys)
   "Generate a stream of package expressions for PACKAGE-NAME and all its
 dependencies."
-  (receive (package . dependencies)
-      (repo->guix-package package-name repo)
-    (if (not package)
-        stream-null
-
-        ;; Generate a lazy stream of package expressions for all unknown
-        ;; dependencies in the graph.
-        (let* ((make-state (lambda (queue done)
-                             (cons queue done)))
-               (next       (match-lambda
-                             (((next . rest) . done) next)))
-               (imported   (match-lambda
-                             ((queue . done) done)))
-               (done?      (match-lambda
-                             ((queue . done)
-                              (zero? (length queue)))))
-               (unknown?   (lambda* (dependency #:optional (done '()))
-                             (and (not (member dependency
-                                               done))
-                                  (null? (find-packages-by-name
-                                          (guix-name dependency))))))
-               (update     (lambda (state new-queue)
-                             (match state
-                               (((head . tail) . done)
-                                (make-state (lset-difference
-                                             equal?
-                                             (lset-union equal? new-queue tail)
-                                             done)
-                                            (cons head done)))))))
-          (stream-cons
-           package
-           (stream-unfold
-            ;; map: produce a stream element
-            (lambda (state)
-              (repo->guix-package (next state) repo))
-
-            ;; predicate
-            (negate done?)
-
-            ;; generator: update the queue
-            (lambda (state)
-              (receive (package . dependencies)
-                  (repo->guix-package (next state) repo)
-                (if package
-                    (update state (filter (cut unknown? <>
-                                               (cons (next state)
-                                                     (imported state)))
-                                          (car dependencies)))
-                    ;; TODO: Try the other archives before giving up
-                    (update state (imported state)))))
-
-            ;; initial state
-            (make-state (filter unknown? (car dependencies))
-                        (list package-name))))))))
+  (define (exists? dependency)
+    (not (null? (find-packages-by-name (guix-name dependency)))))
+  (define initial-state (list #f (list package-name) (list)))
+  (define (step state)
+    (match state
+      ((prev (next . rest) done)
+       (define (handle? dep)
+         (and
+           (not (equal? dep next))
+           (not (member dep done))
+           (not (exists? dep))))
+       (receive (package . dependencies) (repo->guix-package next repo)
+         (list
+           (if package package '()) ;; default #f on failure would interrupt
+           (if package
+             (lset-union equal? rest (filter handle? (car dependencies)))
+             rest)
+           (cons next done))))
+      ((prev '() done)
+       (list #f '() done))))
+
+  ;; Generate a lazy stream of package expressions for all unknown
+  ;; dependencies in the graph.
+  (stream-unfold
+    ;; map: produce a stream element
+    (match-lambda ((latest queue done) latest))
+    ;; predicate
+    (match-lambda ((latest queue done) latest))
+    ;; generator: update the queue
+    step
+    ;; initial state
+    (step initial-state)))
-- 
2.20.1 (Apple Git-117)

[bug#35935] [PATCH] guix: import: simplify recursive import

[Ludovic Courtès]

Hello Robert,

Robert Vollmert <rob@vllmrt.net> skribis:

> This simplifies the logic of recursive-import, intending no
> major functional changes. The package import function is no
> longer called twice per package. Failed imports now make it
> to the package stream as '() instead of #f.
>
> * guix/import/utils.scm: Simplify recursive-import.
                         ^
Minor issue: please make sure to mention the modified entities here
(procedures, variables, etc.); see
<https://www.gnu.org/prep/standards/html_node/Change-Logs.html>.

I think Oleg worked on this part before; Oleg, could you comment
and/or apply?

‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
perhaps that’s something we should fix eventually.

Thanks,
Ludo’.

[bug#35935] [PATCH] guix: import: simplify recursive import

[Robert Vollmert]

> On 2. Jun 2019, at 21:52, Ludovic Courtès <ludo@gnu.org> wrote:
> Robert Vollmert <rob@vllmrt.net> skribis:
>> * guix/import/utils.scm: Simplify recursive-import.
>                         ^
> Minor issue: please make sure to mention the modified entities here
> (procedures, variables, etc.); see
> <https://www.gnu.org/prep/standards/html_node/Change-Logs.html>.

Fixed, thanks for the heads up.

> I think Oleg worked on this part before; Oleg, could you comment
> and/or apply?
> 
> ‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
> perhaps that’s something we should fix eventually.

Agreed. It does seem to be covered somewhat by tests/gem.scm.
I also tested it a bit by hand, with some haskell packages.

[bug#35935] [PATCH] guix: import: simplify recursive import

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

Hello Robert,

Robert Vollmert <rob@vllmrt.net> writes:

[…]

>> I think Oleg worked on this part before; Oleg, could you comment
>> and/or apply?
>> 
>> ‘tests/import-utils.scm’ doesn’t seem to be testing this procedure,
>> perhaps that’s something we should fix eventually.
>
> Agreed. It does seem to be covered somewhat by tests/gem.scm.
> I also tested it a bit by hand, with some haskell packages.

True, ‘tests/gem.scm’ file tests ‘recursive-import’ procedure by
invoking ‘gem-recursive-import’.  I think this test is good enough for
our purpose - use ‘recursive-import’ in other package importers and make
sure it works. Thoughts?

I also added a copyright line if you don't mind:

    ;;; Copyright © 2019 Robert Vollmert <rob@vllmrt.net>

Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
importers and ready to push :-)

Thanks,
Oleg.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#35935] [PATCH] guix: import: simplify recursive import

[Robert Vollmert]

Hello Oleg,

thanks for having a look.

> On 3. Jun 2019, at 21:56, Oleg Pykhalov <go.wigust@gmail.com> wrote:

[…]

> I also added a copyright line if you don't mind:
> 
>    ;;; Copyright © 2019 Robert Vollmert <rob@vllmrt.net>
> 
> Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
> importers and ready to push :-)

Yes, of course. (Should I generally be adding copyright lines when edit
a file?

Cheers
Robert

bug#35935: [PATCH] guix: import: simplify recursive import

[Oleg Pykhalov]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

Robert Vollmert <rob@vllmrt.net> writes:

[…]

>> I also added a copyright line if you don't mind:
>> 
>>    ;;; Copyright © 2019 Robert Vollmert <rob@vllmrt.net>
>> 
>> Is it OK, Robert?  I tested manually with ‘gem’ and ‘elpa’ recursive
>> importers and ready to push :-)
>
> Yes, of course. (Should I generally be adding copyright lines when edit
> a file?

Yes.

Pushed as 5b315f3ea93020df52bc11105064a1398687e572 to master.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#35935] [PATCH] guix: import: Simplify recursive import.

[Robert Vollmert]

This simplifies the logic of recursive-import, intending no
major functional changes. The package import function is no
longer called twice per package. Failed imports now make it
to the package stream as '() instead of #f.

* guix/import/utils.scm (recursive-import): Simplify.
---
 guix/import/utils.scm | 86 ++++++++++++++++---------------------------
 1 file changed, 32 insertions(+), 54 deletions(-)

diff --git a/guix/import/utils.scm b/guix/import/utils.scm
index 516c0cfaa2..ff548b809a 100644
--- a/guix/import/utils.scm
+++ b/guix/import/utils.scm
@@ -378,57 +378,35 @@ separated by PRED."
                            #:allow-other-keys)
   "Generate a stream of package expressions for PACKAGE-NAME and all its
 dependencies."
-  (receive (package . dependencies)
-      (repo->guix-package package-name repo)
-    (if (not package)
-        stream-null
-
-        ;; Generate a lazy stream of package expressions for all unknown
-        ;; dependencies in the graph.
-        (let* ((make-state (lambda (queue done)
-                             (cons queue done)))
-               (next       (match-lambda
-                             (((next . rest) . done) next)))
-               (imported   (match-lambda
-                             ((queue . done) done)))
-               (done?      (match-lambda
-                             ((queue . done)
-                              (zero? (length queue)))))
-               (unknown?   (lambda* (dependency #:optional (done '()))
-                             (and (not (member dependency
-                                               done))
-                                  (null? (find-packages-by-name
-                                          (guix-name dependency))))))
-               (update     (lambda (state new-queue)
-                             (match state
-                               (((head . tail) . done)
-                                (make-state (lset-difference
-                                             equal?
-                                             (lset-union equal? new-queue tail)
-                                             done)
-                                            (cons head done)))))))
-          (stream-cons
-           package
-           (stream-unfold
-            ;; map: produce a stream element
-            (lambda (state)
-              (repo->guix-package (next state) repo))
-
-            ;; predicate
-            (negate done?)
-
-            ;; generator: update the queue
-            (lambda (state)
-              (receive (package . dependencies)
-                  (repo->guix-package (next state) repo)
-                (if package
-                    (update state (filter (cut unknown? <>
-                                               (cons (next state)
-                                                     (imported state)))
-                                          (car dependencies)))
-                    ;; TODO: Try the other archives before giving up
-                    (update state (imported state)))))
-
-            ;; initial state
-            (make-state (filter unknown? (car dependencies))
-                        (list package-name))))))))
+  (define (exists? dependency)
+    (not (null? (find-packages-by-name (guix-name dependency)))))
+  (define initial-state (list #f (list package-name) (list)))
+  (define (step state)
+    (match state
+      ((prev (next . rest) done)
+       (define (handle? dep)
+         (and
+           (not (equal? dep next))
+           (not (member dep done))
+           (not (exists? dep))))
+       (receive (package . dependencies) (repo->guix-package next repo)
+         (list
+           (if package package '()) ;; default #f on failure would interrupt
+           (if package
+             (lset-union equal? rest (filter handle? (car dependencies)))
+             rest)
+           (cons next done))))
+      ((prev '() done)
+       (list #f '() done))))
+
+  ;; Generate a lazy stream of package expressions for all unknown
+  ;; dependencies in the graph.
+  (stream-unfold
+    ;; map: produce a stream element
+    (match-lambda ((latest queue done) latest))
+    ;; predicate
+    (match-lambda ((latest queue done) latest))
+    ;; generator: update the queue
+    step
+    ;; initial state
+    (step initial-state)))
-- 
2.20.1 (Apple Git-117)