bug#26652: [PATCH 0/2] fresh: add -m manifest option.

bug#26652: [PATCH 0/2] fresh: add -m manifest option.

[Mathieu Othacehe]

Hi,

Here's a small serie to add -m manifest support to guix refresh.
It allows to find, selfishly, you packages not up-to-date.

Thanks,

Mathieu

Mathieu Othacehe (2):
  scripts: refresh: Add -m manifest option.
  manifest dependencies

 guix/scripts/refresh.scm | 78 ++++++++++++++++++++++++++++++++++++------------
 1 file changed, 59 insertions(+), 19 deletions(-)

-- 
2.12.2

bug#26652: [PATCH 1/2] profiles: Add manifest->packages.

[Mathieu Othacehe]

* guix/profiles.scm (manifest->packages): New exported procedure.
---
 guix/profiles.scm | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/guix/profiles.scm b/guix/profiles.scm
index eb172ef45..157445407 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -8,6 +8,7 @@
 ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2017 Huang Ying <huang.ying.caritas@gmail.com>
 ;;; Copyright © 2017 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -94,6 +95,7 @@
             profile-manifest
             package->manifest-entry
             packages->manifest
+            manifest->packages
             ca-certificate-bundle
             %default-profile-hooks
             profile-derivation
@@ -203,6 +205,14 @@ denoting a specific output of a package."
           (package->manifest-entry package)))
         packages)))
 
+(define (manifest->packages manifest)
+  "Return the list of packages in MANIFEST."
+  (delete-duplicates
+   (filter-map (lambda (entry)
+                 (let ((item (manifest-entry-item entry)))
+                   (if (package? item) item #f)))
+               (manifest-entries manifest))))
+
 (define (manifest->gexp manifest)
   "Return a representation of MANIFEST as a gexp."
   (define (entry->gexp entry)
-- 
2.12.2

bug#26652: [PATCH 2/2] scripts: refresh: Add -m manifest option.

[Mathieu Othacehe]

* guix/scripts/refresh.scm (%options): Add -m option,
(show-help): document it,
(get-manifest-packages): new procedure,
(guix-refresh): use packages from manifest if specified, otherwise
keep the previous behaviour.
---
 guix/scripts/refresh.scm | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 4d3c695aa..36e339107 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2015 Alex Kost <alezost@gmail.com>
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -27,6 +28,7 @@
   #:use-module (guix store)
   #:use-module (guix utils)
   #:use-module (guix packages)
+  #:use-module (guix profiles)
   #:use-module (guix upstream)
   #:use-module (guix graph)
   #:use-module (guix scripts graph)
@@ -87,6 +89,9 @@
         (option '(#\L "list-updaters") #f #f
                 (lambda args
                   (list-updaters-and-exit)))
+        (option '(#\m "manifest") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'manifest arg result)))
         (option '(#\e "expression") #t #f
                 (lambda (opt name arg result)
                   (alist-cons 'expression arg result)))
@@ -133,6 +138,8 @@ specified with `--select'.\n"))
   -s, --select=SUBSET    select all the packages in SUBSET, one of
                          `core' or `non-core'"))
   (display (_ "
+  -m, --manifest=FILE    select all the packages in FILE manifest"))
+  (display (_ "
   -t, --type=UPDATER,... restrict to updates from the specified updaters
                          (e.g., 'gnu')"))
   (display (_ "
@@ -346,6 +353,17 @@ dependent packages are rebuilt: ~{~a~^ ~}~%"
 
 \f
 ;;;
+;;; Manifest.
+;;;
+
+(define (get-manifest-packages manifest)
+  "Return the list of packages in loaded MANIFEST."
+  (let* ((user-module (make-user-module '((guix profiles) (gnu))))
+         (manifest    (load* manifest user-module)))
+    (manifest->packages manifest)))
+
+\f
+;;;
 ;;; Entry point.
 ;;;
 
@@ -416,8 +434,11 @@ update would trigger a complete rebuild."
          ;; the command line.
          (warn?           (or (assoc-ref opts 'argument)
                               (assoc-ref opts 'expression)))
-
-         (packages
+         (manifest-packages
+          (match (assoc-ref opts 'manifest)
+            ((? string? manifest) (get-manifest-packages manifest))
+            (x                    #f)))
+         (args-packages
           (match (filter-map (match-lambda
                                (('argument . spec)
                                 ;; Take either the specified version or the
@@ -438,7 +459,9 @@ update would trigger a complete rebuild."
                                     result))
                               '())))
             (some                                 ; user-specified packages
-             some))))
+             some)))
+         (packages
+          (or manifest-packages args-packages)))
     (with-error-handling
       (with-store store
         (run-with-store store
-- 
2.12.2

bug#26652: [PATCH 1/2] profiles: Add manifest->packages.

[Maxim Cournoyer]

Hi Mathieu!

Mathieu Othacehe <m.othacehe@gmail.com> writes:

> * guix/profiles.scm (manifest->packages): New exported procedure.
> ---
>  guix/profiles.scm | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/guix/profiles.scm b/guix/profiles.scm
> index eb172ef45..157445407 100644
> --- a/guix/profiles.scm
> +++ b/guix/profiles.scm
> @@ -8,6 +8,7 @@
>  ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
>  ;;; Copyright © 2017 Huang Ying <huang.ying.caritas@gmail.com>
>  ;;; Copyright © 2017 Maxim Cournoyer <maxim.cournoyer@gmail.com>
> +;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -94,6 +95,7 @@
>              profile-manifest
>              package->manifest-entry
>              packages->manifest
> +            manifest->packages
>              ca-certificate-bundle
>              %default-profile-hooks
>              profile-derivation
> @@ -203,6 +205,14 @@ denoting a specific output of a package."
>            (package->manifest-entry package)))
>          packages)))
>  
> +(define (manifest->packages manifest)
> +  "Return the list of packages in MANIFEST."
> +  (delete-duplicates
> +   (filter-map (lambda (entry)
> +                 (let ((item (manifest-entry-item entry)))
> +                   (if (package? item) item #f)))
> +               (manifest-entries manifest))))
> +
>  (define (manifest->gexp manifest)
>    "Return a representation of MANIFEST as a gexp."
>    (define (entry->gexp entry)

The filter-map part seems problematic, IIUC. You might end up with an
`#f' in your list of packages, no? Maybe you could use something like:

(delete-duplicates
 (filter package? (map manifest-entry-item (manifest-entries manifest))))

Disclaimer: I haven't tested it.

Maxim

bug#26652: [PATCH 1/2] profiles: Add manifest->packages.

[Mathieu Othacehe]

Hi Maxim,

> The filter-map part seems problematic, IIUC. You might end up with an
> `#f' in your list of packages, no? Maybe you could use something like:

Your variant seems easier to understand, but it seems to me that
filter-map removes #f applications of the list.

Thanks,

Mathieu

bug#26652: [PATCH 1/2] profiles: Add manifest->packages.

[Maxim Cournoyer]

On April 25, 2017 10:00:04 AM PDT, Mathieu Othacehe <m.othacehe@gmail.com> wrote:
>
>Hi Maxim,
>
>> The filter-map part seems problematic, IIUC. You might end up with an
>> `#f' in your list of packages, no? Maybe you could use something
>like:
>
>Your variant seems easier to understand, but it seems to me that
>filter-map removes #f applications of the list.
>

Oh, right! It seems I failed to see the effect of `filter' in `filter-map' ;)

Maxim

bug#26652: [PATCH 1/2] profiles: Add manifest->packages.

[Ludovic Courtès]

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> * guix/profiles.scm (manifest->packages): New exported procedure.
> ---
>  guix/profiles.scm | 10 ++++++++++
>  1 file changed, 10 insertions(+)
>
> diff --git a/guix/profiles.scm b/guix/profiles.scm
> index eb172ef45..157445407 100644
> --- a/guix/profiles.scm
> +++ b/guix/profiles.scm
> @@ -8,6 +8,7 @@
>  ;;; Copyright © 2016 Chris Marusich <cmmarusich@gmail.com>
>  ;;; Copyright © 2017 Huang Ying <huang.ying.caritas@gmail.com>
>  ;;; Copyright © 2017 Maxim Cournoyer <maxim.cournoyer@gmail.com>
> +;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -94,6 +95,7 @@
>              profile-manifest
>              package->manifest-entry
>              packages->manifest
> +            manifest->packages
>              ca-certificate-bundle
>              %default-profile-hooks
>              profile-derivation
> @@ -203,6 +205,14 @@ denoting a specific output of a package."
>            (package->manifest-entry package)))
>          packages)))
>  
> +(define (manifest->packages manifest)
> +  "Return the list of packages in MANIFEST."
> +  (delete-duplicates
> +   (filter-map (lambda (entry)
> +                 (let ((item (manifest-entry-item entry)))
> +                   (if (package? item) item #f)))
> +               (manifest-entries manifest))))

You can remove ‘delete-duplicates’ IMO.

Since this procedure has special semantics (it omits non-package-object
entries), what about keeping it private in (guix scripts refresh)?

Ludo’.

bug#26652: [PATCH 2/2] scripts: refresh: Add -m manifest option.

[Ludovic Courtès]

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> * guix/scripts/refresh.scm (%options): Add -m option,
> (show-help): document it,
> (get-manifest-packages): new procedure,
> (guix-refresh): use packages from manifest if specified, otherwise
> keep the previous behaviour.

This sounds useful!

>    (display (_ "
> +  -m, --manifest=FILE    select all the packages in FILE manifest"))

“select all the package from the manifest in FILE” maybe?

> +(define (get-manifest-packages manifest)
> +  "Return the list of packages in loaded MANIFEST."
> +  (let* ((user-module (make-user-module '((guix profiles) (gnu))))
> +         (manifest    (load* manifest user-module)))
> +    (manifest->packages manifest)))

With ‘manifest->packages’ moved in this module, that would be fine.

Also, maybe s/get-manifest-packages/packages-from-manifest/, because
‘get-’ is quite unusual.  :-)

> +         (packages
> +          (or manifest-packages args-packages)))

Maybe (append (match (assoc-ref opts 'manifest)
                (#f '())
                ((? string? file) (packages-from-manifest file)))
              args-packages)

?

Could you also update guix.texi to mention this option and the intended
use case?

Thank you!

Ludo’.

bug#26652: [PATCH] scripts: refresh: Add -m manifest option.

[Mathieu Othacehe]

* guix/scripts/refresh.scm (%options): Add -m option,
(show-help): document it,
(packages-from-manifest): new procedure,
(guix-refresh): use packages from manifest if specified, otherwise
keep the previous behaviour.
* doc/guix.texi (Invoking guix refresh): document new option.
---
 doc/guix.texi            |  5 +++++
 guix/scripts/refresh.scm | 34 +++++++++++++++++++++++++++++++---
 2 files changed, 36 insertions(+), 3 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 9b2fe3fdb..0c07ffb22 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5759,6 +5759,11 @@ The @code{non-core} subset refers to the remaining packages.  It is
 typically useful in cases where an update of the core packages would be
 inconvenient.
 
+@item --manifest=@var{file}
+@itemx -m @var{file}
+Select all the packages from the manifest in @var{file}. This is useful to
+check if any packages of the user manifest can be updated.
+
 @item --type=@var{updater}
 @itemx -t @var{updater}
 Select only packages handled by @var{updater} (may be a comma-separated
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 645572f18..e22661178 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -4,6 +4,7 @@
 ;;; Copyright © 2014 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2015 Alex Kost <alezost@gmail.com>
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
+;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -27,6 +28,7 @@
   #:use-module (guix store)
   #:use-module (guix utils)
   #:use-module (guix packages)
+  #:use-module (guix profiles)
   #:use-module (guix upstream)
   #:use-module (guix graph)
   #:use-module (guix scripts graph)
@@ -87,6 +89,9 @@
         (option '(#\L "list-updaters") #f #f
                 (lambda args
                   (list-updaters-and-exit)))
+        (option '(#\m "manifest") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'manifest arg result)))
         (option '(#\e "expression") #t #f
                 (lambda (opt name arg result)
                   (alist-cons 'expression arg result)))
@@ -133,6 +138,8 @@ specified with `--select'.\n"))
   -s, --select=SUBSET    select all the packages in SUBSET, one of
                          `core' or `non-core'"))
   (display (G_ "
+  -m, --manifest=FILE    select all the packages from the manifest in FILE"))
+  (display (G_ "
   -t, --type=UPDATER,... restrict to updates from the specified updaters
                          (e.g., 'gnu')"))
   (display (G_ "
@@ -350,6 +357,24 @@ dependent packages are rebuilt: ~{~a~^ ~}~%"
 
 \f
 ;;;
+;;; Manifest.
+;;;
+
+(define (manifest->packages manifest)
+  "Return the list of packages in MANIFEST."
+  (filter-map (lambda (entry)
+                (let ((item (manifest-entry-item entry)))
+                  (if (package? item) item #f)))
+              (manifest-entries manifest)))
+
+(define (packages-from-manifest manifest)
+  "Return the list of packages in loaded MANIFEST."
+  (let* ((user-module (make-user-module '((guix profiles) (gnu))))
+         (manifest    (load* manifest user-module)))
+    (manifest->packages manifest)))
+
+\f
+;;;
 ;;; Entry point.
 ;;;
 
@@ -420,8 +445,7 @@ update would trigger a complete rebuild."
          ;; the command line.
          (warn?           (or (assoc-ref opts 'argument)
                               (assoc-ref opts 'expression)))
-
-         (packages
+         (args-packages
           (match (filter-map (match-lambda
                                (('argument . spec)
                                 ;; Take either the specified version or the
@@ -442,7 +466,11 @@ update would trigger a complete rebuild."
                                     result))
                               '())))
             (some                                 ; user-specified packages
-             some))))
+             some)))
+         (packages
+          (match (assoc-ref opts 'manifest)
+            (#f args-packages)
+            ((? string? file) (packages-from-manifest file)))))
     (with-error-handling
       (with-store store
         (run-with-store store
-- 
2.12.2

bug#26652: [PATCH 2/2] scripts: refresh: Add -m manifest option.

[Mathieu Othacehe]

Hi Ludo,

Thanks for the review :)

>>    (display (_ "
>> +  -m, --manifest=FILE    select all the packages in FILE manifest"))
>
> “select all the package from the manifest in FILE” maybe?

Ok.

> Also, maybe s/get-manifest-packages/packages-from-manifest/, because
> ‘get-’ is quite unusual.  :-)

Seems better !

>
>> +         (packages
>> +          (or manifest-packages args-packages)))
>
> Maybe (append (match (assoc-ref opts 'manifest)
>                 (#f '())
>                 ((? string? file) (packages-from-manifest file)))
>               args-packages)

Mmh, I dont think we want to append manifest packages and args-packages.
We want only manifest packages if a manifest is specified IMO.

>
> ?
>
> Could you also update guix.texi to mention this option and the intended
> use case?

Sure.

I sent an updated patch taking your remarks into account.

Mathieu

bug#26652: [PATCH] scripts: refresh: Add -m manifest option.

[Ludovic Courtès]

Hi Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> * guix/scripts/refresh.scm (%options): Add -m option,
> (show-help): document it,
> (packages-from-manifest): new procedure,
> (guix-refresh): use packages from manifest if specified, otherwise
> keep the previous behaviour.
> * doc/guix.texi (Invoking guix refresh): document new option.

This revised patch had fallen through the cracks, but it LGTM!
Please push.

Thanks for your patience.  :-)

Ludo’.

bug#26652: [PATCH] scripts: refresh: Add -m manifest option.

[Mathieu Othacehe]

Hi Ludo,

> Thanks for your patience.  :-)

No problem :) Pushed as 1335ac314.

Thanks,

Mathieu