unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#32115] [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l'
@ 2018-07-10 16:45 Ludovic Courtès
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
  2018-07-13 15:59 ` bug#32115: [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès
  0 siblings, 2 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:45 UTC (permalink / raw)
  To: 32115

Hello Guix!

This patch adds (guix inferior), a module to interact with an inferior Guix
process, along with a ‘guix repl’ command, which spawns a REPL optionally a
simple sexp-based protocol.

‘guix pull -l’ uses (guix inferior) to display this:

--8<---------------cut here---------------start------------->8---
Generation 1	Jun 10 2018 00:18:18
  guix 65956ad
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: origin/master
    commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe
Generation 2	Jun 11 2018 11:02:49
  guix e0cc7f6
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: origin/master
    commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d
  2 new packages: keepalived, libnfnetlink
  6 packages upgraded: emacs-nix-mode@2.0.4, guile2.0-guix@0.14.0-12.77a1aac,
    guix@0.14.0-12.77a1aac, heimdal@7.5.0, milkytracker@1.02.00, nix@2.0.4
Generation 3	Jun 13 2018 23:31:07
  guix 844cc1c
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: origin/master
    commit: 844cc1c8f394f03b404c5bb3aee086922373490c
  28 new packages: emacs-helm-ls-git, emacs-helm-mu, emacs-helm-pass, emacs-image+,
    emacs-package-lint, emacs-puppet-mode, emacs-wgrep-helm, go-github-com-gorilla-mux,
    go-github-com-jonboulle-clockwork, go-github-com-spf13-pflag, libostree, ovmf-aarch64, ovmf-arm, qtwebglplugin,
    qtwebview, r-ellipse, r-factoextra, r-factominer, r-flashclust, r-ggpubr, r-ggsci, r-ggsignif, r-hdf5r,
    r-nbclust, r-writexl, skopeo, umoci, vkd3d
  69 packages upgraded: borg@1.1.6, cheese@3.28.0, cpupower@4.17.1, efivar@36, emacs-orgalist@1.8,
    emacspeak@48.0, font-gnu-unifont@11.0.01, freefall@4.17.1, glslang@3.0-3.32d3ec319, guile-bytestructures@1.0.3,
    guile2.0-bytestructures@1.0.3, iproute2@4.17.0, ldb@1.4.0, libfilezilla@0.12.3, libndp@1.7, libraw@0.18.12,

[...]

--8<---------------cut here---------------end--------------->8---

It takes about one second per generation on my SSD-powered laptop, though, so
we might want to have a persistent cache of each generation’s package list.

Eventually I think we can use (guix inferior) for other purposes.  For example,
hpcguix-web should use it to regularly grab the latest package list.  And, we
could add a gexp compiler for <inferior-package> such that one can seamlessly
refer to packages coming from a different Guix version.  Then, if we wanted to,
we could have ‘guix package’ & co. allow you to specify packages coming from
a different Guix.  Could be fun!

Thoughts?  Comments?

Ludo’.

Ludovic Courtès (2):
  Add (guix inferior) and (guix scripts repl).
  pull: Use (guix inferior) to display new and upgraded packages.

 Makefile.am           |   3 +
 doc/guix.texi         |  59 +++++++++++++
 guix/inferior.scm     | 197 +++++++++++++++++++++++++++++++++++++++++
 guix/scripts/pull.scm |  91 ++++++++++++++++---
 guix/scripts/repl.scm | 199 ++++++++++++++++++++++++++++++++++++++++++
 tests/inferior.scm    |  69 +++++++++++++++
 6 files changed, 607 insertions(+), 11 deletions(-)
 create mode 100644 guix/inferior.scm
 create mode 100644 guix/scripts/repl.scm
 create mode 100644 tests/inferior.scm

-- 
2.18.0

^ permalink raw reply	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl).
  2018-07-10 16:45 [bug#32115] [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès
@ 2018-07-10 16:48 ` Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 1/2] Add (guix inferior) Ludovic Courtès
                     ` (5 more replies)
  2018-07-13 15:59 ` bug#32115: [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès
  1 sibling, 6 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/inferior.scm, guix/scripts/repl.scm, tests/inferior.scm: New
files.
* Makefile.am (MODULES): Add 'guix/scripts/repl.scm' and
'guix/inferior.scm'.
(SCM_TESTS): Add 'tests/inferior.scm'.
* doc/guix.texi (Invoking guix repl): New node.
---
 Makefile.am           |   3 +
 doc/guix.texi         |  53 +++++++++++
 guix/inferior.scm     | 197 +++++++++++++++++++++++++++++++++++++++++
 guix/scripts/repl.scm | 199 ++++++++++++++++++++++++++++++++++++++++++
 tests/inferior.scm    |  69 +++++++++++++++
 5 files changed, 521 insertions(+)
 create mode 100644 guix/inferior.scm
 create mode 100644 guix/scripts/repl.scm
 create mode 100644 tests/inferior.scm

diff --git a/Makefile.am b/Makefile.am
index 5dc04de35..8d05744bc 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -85,6 +85,7 @@ MODULES =					\
   guix/nar.scm					\
   guix/derivations.scm				\
   guix/grafts.scm				\
+  guix/inferior.scm				\
   guix/gnu-maintenance.scm			\
   guix/self.scm					\
   guix/upstream.scm				\
@@ -200,6 +201,7 @@ MODULES =					\
   guix/scripts/substitute.scm			\
   guix/scripts/authenticate.scm			\
   guix/scripts/refresh.scm			\
+  guix/scripts/repl.scm				\
   guix/scripts/system.scm			\
   guix/scripts/system/search.scm		\
   guix/scripts/lint.scm				\
@@ -355,6 +357,7 @@ SCM_TESTS =					\
   tests/profiles.scm				\
   tests/search-paths.scm			\
   tests/syscalls.scm				\
+  tests/inferior.scm				\
   tests/gremlin.scm				\
   tests/bournish.scm				\
   tests/lint.scm				\
diff --git a/doc/guix.texi b/doc/guix.texi
index 7ce364b0a..e93b320e8 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -166,6 +166,7 @@ Programming Interface
 * Derivations::                 Low-level interface to package derivations.
 * The Store Monad::             Purely functional interface to the store.
 * G-Expressions::               Manipulating build expressions.
+* Invoking guix repl::          Fiddling with Guix interactively.
 
 Defining Packages
 
@@ -3267,6 +3268,7 @@ package definitions.
 * Derivations::                 Low-level interface to package derivations.
 * The Store Monad::             Purely functional interface to the store.
 * G-Expressions::               Manipulating build expressions.
+* Invoking guix repl::          Fiddling with Guix interactively.
 @end menu
 
 @node Defining Packages
@@ -5538,6 +5540,57 @@ corresponding to @var{obj} for @var{system}, cross-compiling for
 has an associated gexp compiler, such as a @code{<package>}.
 @end deffn
 
+@node Invoking guix repl
+@section Invoking @command{guix repl}
+
+@cindex REPL, read-eval-print loop
+The @command{guix repl} command spawns a Guile @dfn{read-eval-print loop}
+(REPL) for interactive programming (@pxref{Using Guile Interactively,,, guile,
+GNU Guile Reference Manual}).  Compared to just launching the @command{guile}
+command, @command{guix repl} guarantees that all the Guix modules and all its
+dependencies are available in the search path.  You can use it this way:
+
+@example
+$ guix repl
+scheme@@(guile-user)> ,use (gnu packages base)
+scheme@@(guile-user)> coreutils
+$1 = #<package coreutils@@8.29 gnu/packages/base.scm:327 3e28300>
+@end example
+
+@cindex inferiors
+In addition, @command{guix repl} implements a simple machine-readable REPL
+protocol for use by @code{(guix inferior)}, a facility to interact with
+@dfn{inferiors}, separate processes running a potentially different revision
+of Guix.
+
+The available options are as follows:
+
+@table @code
+@item --type=@var{type}
+@itemx -t @var{type}
+Start a REPL of the given @var{TYPE}, which can be one of the following:
+
+@table @code
+@item guile
+This is default, and it spawns a standard full-featured Guile REPL.
+@item machine
+Spawn a REPL that uses the machine-readable protocol.  This is the protocol
+that the @code{(guix inferior)} module speaks.
+@end table
+
+@item --listen=@var{endpoint}
+By default, @command{guix repl} reads from standard input and writes to
+standard output.  When this option is passed, it will instead listen for
+connections on @var{endpoint}.  Here are examples of valid options:
+
+@table @code
+@item --listen=tcp:37146
+Accept connections on localhost on port 37146.
+
+@item --listen=unix:/tmp/socket
+Accept connections on the Unix-domain socket @file{/tmp/socket}.
+@end table
+@end table
 
 @c *********************************************************************
 @node Utilities
diff --git a/guix/inferior.scm b/guix/inferior.scm
new file mode 100644
index 000000000..629c2c431
--- /dev/null
+++ b/guix/inferior.scm
@@ -0,0 +1,197 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix inferior)
+  #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 popen)
+  #:export (inferior?
+            open-inferior
+            close-inferior
+            inferior-eval
+            inferior-object?
+
+            inferior-package?
+            inferior-package-name
+            inferior-package-version
+
+            inferior-packages
+            inferior-package-synopsis
+            inferior-package-description))
+
+;;; Commentary:
+;;;
+;;; This module provides a way to spawn Guix "inferior" processes and to talk
+;;; to them.  It allows us, from one instance of Guix, to interact with
+;;; another instance of Guix coming from a different commit.
+;;;
+;;; Code:
+
+;; Inferior Guix process.
+(define-record-type <inferior>
+  (inferior pid socket version)
+  inferior?
+  (pid      inferior-pid)
+  (socket   inferior-socket)
+  (version  inferior-version))                    ;REPL protocol version
+
+(define (inferior-pipe directory command)
+  "Return an input/output pipe on the Guix instance in DIRECTORY.  This runs
+'DIRECTORY/COMMAND repl' if it exists, or falls back to some other method if
+it's an old Guix."
+  (let ((pipe (with-error-to-port (%make-void-port "w")
+                (lambda ()
+                  (open-pipe* OPEN_BOTH
+                              (string-append directory "/" command)
+                              "repl" "-t" "machine")))))
+    (if (eof-object? (peek-char pipe))
+        (begin
+          (close-pipe pipe)
+
+          ;; Older versions of Guix didn't have a 'guix repl' command, so
+          ;; emulate it.
+          (open-pipe* OPEN_BOTH "guile"
+                      "-L" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/lib/guile/"
+                                          (effective-version) "/site-ccache")
+                      "-c"
+                      (object->string
+                       `(begin
+                          (primitive-load ,(search-path %load-path
+                                                        "guix/scripts/repl.scm"))
+                          ((@ (guix scripts repl) machine-repl))))))
+        pipe)))
+
+(define* (open-inferior directory #:key (command "bin/guix"))
+  "Open the inferior Guix in DIRECTORY, running 'DIRECTORY/COMMAND repl' or
+equivalent.  Return #f if the inferior could not be launched."
+  (define pipe
+    (inferior-pipe directory command))
+
+  (setvbuf pipe _IOLBF)
+  (match (read pipe)
+    (('repl-version 0 rest ...)
+     (let ((result (inferior 'pipe pipe (cons 0 rest))))
+       (inferior-eval '(use-modules (guix)) result)
+       (inferior-eval '(use-modules (gnu)) result)
+       (inferior-eval '(define %package-table (make-hash-table))
+                      result)
+       result))
+    (_
+     #f)))
+
+(define (close-inferior inferior)
+  "Close INFERIOR."
+  (close-pipe (inferior-socket inferior)))
+
+;; Non-self-quoting object of the inferior.
+(define-record-type <inferior-object>
+  (inferior-object address appearance)
+  inferior-object?
+  (address     inferior-object-address)
+  (appearance  inferior-object-appearance))
+
+(define (write-inferior-object object port)
+  (match object
+    (($ <inferior-object> _ appearance)
+     (format port "#<inferior-object ~a>" appearance))))
+
+(set-record-type-printer! <inferior-object> write-inferior-object)
+
+(define (inferior-eval exp inferior)
+  "Evaluate EXP in INFERIOR."
+  (define sexp->object
+    (match-lambda
+      (('value value)
+       value)
+      (('non-self-quoting address string)
+       (inferior-object address string))))
+
+  (write exp (inferior-socket inferior))
+  (newline (inferior-socket inferior))
+  (match (read (inferior-socket inferior))
+    (('values objects ...)
+     (apply values (map sexp->object objects)))
+    (('exception key objects ...)
+     (apply throw key (map sexp->object objects)))))
+
+\f
+;;;
+;;; Inferior packages.
+;;;
+
+(define-record-type <inferior-package>
+  (inferior-package inferior name version id)
+  inferior-package?
+  (inferior   inferior-package-inferior)
+  (name       inferior-package-name)
+  (version    inferior-package-version)
+  (id         inferior-package-id))
+
+(define (write-inferior-package package port)
+  (match package
+    (($ <inferior-package> _ name version)
+     (format port "#<inferior-package ~a@~a ~a>"
+             name version
+             (number->string (object-address package) 16)))))
+
+(set-record-type-printer! <inferior-package> write-inferior-package)
+
+(define (inferior-packages inferior)
+  "Return the list of packages known to INFERIOR."
+  (let ((result (inferior-eval
+                 '(fold-packages (lambda (package result)
+                                   (let ((id (object-address package)))
+                                     (hashv-set! %package-table id package)
+                                     (cons (list (package-name package)
+                                                 (package-version package)
+                                                 id)
+                                           result)))
+                                 '())
+                 inferior)))
+    (map (match-lambda
+           ((name version id)
+            (inferior-package inferior name version id)))
+         result)))
+
+(define (inferior-package-field package getter)
+  "Return the field of PACKAGE, an inferior package, accessed with GETTER."
+  (let ((inferior (inferior-package-inferior package))
+        (id       (inferior-package-id package)))
+    (inferior-eval `(,getter (hashv-ref %package-table ,id))
+                   inferior)))
+
+(define* (inferior-package-synopsis package #:key (translate? #t))
+  "Return the Texinfo synopsis of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-synopsis)
+                              'package-synopsis)))
+
+(define* (inferior-package-description package #:key (translate? #t))
+  "Return the Texinfo description of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-description)
+                              'package-description)))
diff --git a/guix/scripts/repl.scm b/guix/scripts/repl.scm
new file mode 100644
index 000000000..b157833a4
--- /dev/null
+++ b/guix/scripts/repl.scm
@@ -0,0 +1,199 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix scripts repl)
+  #:use-module (guix ui)
+  #:use-module (guix scripts)
+  #:use-module (guix utils)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-37)
+  #:use-module (ice-9 match)
+  #:use-module (rnrs bytevectors)
+  #:autoload   (system repl repl) (start-repl)
+  #:autoload   (system repl server)
+                  (make-tcp-server-socket make-unix-domain-server-socket)
+  #:export (machine-repl
+            guix-repl))
+
+;;; Commentary:
+;;;
+;;; This command provides a Guile REPL
+
+(define %default-options
+  `((type . guile)))
+
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix repl")))
+        (option '(#\t "type") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'type (string->symbol arg) result)))
+        (option '("listen") #t #f
+                (lambda (opt name arg result)
+                  (alist-cons 'listen arg result)))))
+
+
+(define (show-help)
+  (display (G_ "Usage: guix repl [OPTIONS...]
+Start a Guile REPL in the Guix execution environment.\n"))
+  (display (G_ "
+  -t, --type=TYPE        start a REPL of the given TYPE"))
+  (newline)
+  (display (G_ "
+  -h, --help             display this help and exit"))
+  (display (G_ "
+  -V, --version          display version information and exit"))
+  (newline)
+  (show-bug-report-information))
+
+(define (self-quoting? x)
+  "Return #t if X is self-quoting."
+  (letrec-syntax ((one-of (syntax-rules ()
+                            ((_) #f)
+                            ((_ pred rest ...)
+                             (or (pred x)
+                                 (one-of rest ...))))))
+    (one-of symbol? string? pair? null? vector?
+            bytevector? number? boolean?)))
+
+(define user-module
+  ;; Module where we execute user code.
+  (let ((module (resolve-module '(guix-user) #f #f #:ensure #t)))
+    (beautify-user-module! module)
+    module))
+
+(define* (machine-repl #:optional
+                       (input (current-input-port))
+                       (output (current-output-port)))
+  "Run a machine-usable REPL over ports INPUT and OUTPUT.
+
+The protocol of this REPL is meant to be machine-readable and provides proper
+support to represent multiple-value returns, exceptions, objects that lack a
+read syntax, and so on.  As such it is more convenient and robust than parsing
+Guile's REPL prompt."
+  (define (value->sexp value)
+    (if (self-quoting? value)
+        `(value ,value)
+        `(non-self-quoting ,(object-address value)
+                           ,(object->string value))))
+
+  (write `(repl-version 0 0) output)
+  (newline output)
+  (force-output output)
+
+  (let loop ()
+    (match (read input)
+      ((? eof-object?) #t)
+      (exp
+       (catch #t
+         (lambda ()
+           (let ((results (call-with-values
+                              (lambda ()
+
+                                (primitive-eval exp))
+                            list)))
+             (write `(values ,@(map value->sexp results))
+                    output)
+             (newline output)
+             (force-output output)))
+         (lambda (key . args)
+           (write `(exception ,key ,@(map value->sexp args)))
+           (newline output)
+           (force-output output)))
+       (loop)))))
+
+(define (call-with-connection spec thunk)
+  "Dynamically-bind the current input and output ports according to SPEC and
+call THUNK."
+  (if (not spec)
+      (thunk)
+
+      ;; Note: the "PROTO:" prefix in SPEC is here so that we can eventually
+      ;; parse things like "fd:123" in a non-ambiguous way.
+      (match (string-index spec #\:)
+        (#f
+         (leave (G_ "~A: invalid listen specification~%") spec))
+        (index
+         (let ((protocol (string-take spec index))
+               (address  (string-drop spec (+ index 1))))
+           (define socket
+             (match protocol
+               ("tcp"
+                (make-tcp-server-socket #:port (string->number address)))
+               ("unix"
+                (make-unix-domain-server-socket #:path address))
+               (_
+                (leave (G_ "~A: unsupported protocol family~%")
+                       protocol))))
+
+           (listen socket 10)
+           (let loop ()
+             (match (accept socket)
+               ((connection . address)
+                (if (= AF_UNIX (sockaddr:fam address))
+                    (info (G_ "accepted connection~%"))
+                    (info (G_ "accepted connection from ~a~%")
+                          (inet-ntop (sockaddr:fam address)
+                                     (sockaddr:addr address))))
+                (dynamic-wind
+                  (const #t)
+                  (lambda ()
+                    (parameterize ((current-input-port connection)
+                                   (current-output-port connection))
+                      (thunk)))
+                  (lambda ()
+                    (false-if-exception (close-port connection))
+                    (info (G_ "connection closed~%"))))))
+             (loop)))))))
+
+\f
+(define (guix-repl . args)
+  (define opts
+    ;; Return the list of package names.
+    (args-fold* args %options
+                (lambda (opt name arg result)
+                  (leave (G_ "~A: unrecognized option~%") name))
+                (lambda (arg result)
+                  (leave (G_ "~A: extraneous argument~%") arg))
+                %default-options))
+
+  (with-error-handling
+    (let ((type (assoc-ref opts 'type)))
+      (call-with-connection (assoc-ref opts 'listen)
+        (lambda ()
+          (case type
+            ((guile)
+             (save-module-excursion
+              (lambda ()
+                (set-current-module user-module)
+                (start-repl))))
+            ((machine)
+             (machine-repl))
+            (else
+             (leave (G_ "~a: unknown type of REPL~%") type))))))))
+
+;; Local Variables:
+;; eval: (put 'call-with-connection 'scheme-indent-function 1)
+;; End:
diff --git a/tests/inferior.scm b/tests/inferior.scm
new file mode 100644
index 000000000..5e0f8ae66
--- /dev/null
+++ b/tests/inferior.scm
@@ -0,0 +1,69 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-inferior)
+  #:use-module (guix inferior)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-64))
+
+(define %top-srcdir
+  (dirname (search-path %load-path "guix.scm")))
+
+(define %top-builddir
+  (dirname (search-path %load-compiled-path "guix.go")))
+
+\f
+(test-begin "inferior")
+
+(test-equal "open-inferior"
+  '(42 #t)
+  (let ((inferior (open-inferior %top-builddir
+                                 #:command "scripts/guix")))
+    (and (inferior? inferior)
+         (let ((a (inferior-eval '(apply * '(6 7)) inferior))
+               (b (inferior-eval '(@ (gnu packages base) coreutils)
+                                 inferior)))
+           (close-inferior inferior)
+           (list a (inferior-object? b))))))
+
+(test-equal "inferior-packages"
+  (take (sort (fold-packages (lambda (package lst)
+                               (alist-cons (package-name package)
+                                           (package-version package)
+                                           lst))
+                             '())
+              (lambda (x y)
+                (string<? (car x) (car y))))
+        10)
+  (let* ((inferior (open-inferior %top-builddir
+                                  #:command "scripts/guix"))
+         (packages (inferior-packages inferior)))
+    (and (every string? (map inferior-package-synopsis packages))
+         (begin
+           (close-inferior inferior)
+           (take (sort (map (lambda (package)
+                              (cons (inferior-package-name package)
+                                    (inferior-package-version package)))
+                            packages)
+                       (lambda (x y)
+                         (string<? (car x) (car y))))
+                 10)))))
+
+(test-end "inferior")
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 1/2] Add (guix inferior).
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
@ 2018-07-10 16:48   ` Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 1/3] profiles: Factorize 'manifest-search-paths' Ludovic Courtès
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/inferior.scm, tests/inferior.scm: New files.
---
 Makefile.am        |   2 +
 guix/inferior.scm  | 197 +++++++++++++++++++++++++++++++++++++++++++++
 tests/inferior.scm |  69 ++++++++++++++++
 3 files changed, 268 insertions(+)
 create mode 100644 guix/inferior.scm
 create mode 100644 tests/inferior.scm

diff --git a/Makefile.am b/Makefile.am
index 5dc04de35..25b1d501f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -85,6 +85,7 @@ MODULES =					\
   guix/nar.scm					\
   guix/derivations.scm				\
   guix/grafts.scm				\
+  guix/inferior.scm				\
   guix/gnu-maintenance.scm			\
   guix/self.scm					\
   guix/upstream.scm				\
@@ -355,6 +356,7 @@ SCM_TESTS =					\
   tests/profiles.scm				\
   tests/search-paths.scm			\
   tests/syscalls.scm				\
+  tests/inferior.scm				\
   tests/gremlin.scm				\
   tests/bournish.scm				\
   tests/lint.scm				\
diff --git a/guix/inferior.scm b/guix/inferior.scm
new file mode 100644
index 000000000..629c2c431
--- /dev/null
+++ b/guix/inferior.scm
@@ -0,0 +1,197 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix inferior)
+  #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 popen)
+  #:export (inferior?
+            open-inferior
+            close-inferior
+            inferior-eval
+            inferior-object?
+
+            inferior-package?
+            inferior-package-name
+            inferior-package-version
+
+            inferior-packages
+            inferior-package-synopsis
+            inferior-package-description))
+
+;;; Commentary:
+;;;
+;;; This module provides a way to spawn Guix "inferior" processes and to talk
+;;; to them.  It allows us, from one instance of Guix, to interact with
+;;; another instance of Guix coming from a different commit.
+;;;
+;;; Code:
+
+;; Inferior Guix process.
+(define-record-type <inferior>
+  (inferior pid socket version)
+  inferior?
+  (pid      inferior-pid)
+  (socket   inferior-socket)
+  (version  inferior-version))                    ;REPL protocol version
+
+(define (inferior-pipe directory command)
+  "Return an input/output pipe on the Guix instance in DIRECTORY.  This runs
+'DIRECTORY/COMMAND repl' if it exists, or falls back to some other method if
+it's an old Guix."
+  (let ((pipe (with-error-to-port (%make-void-port "w")
+                (lambda ()
+                  (open-pipe* OPEN_BOTH
+                              (string-append directory "/" command)
+                              "repl" "-t" "machine")))))
+    (if (eof-object? (peek-char pipe))
+        (begin
+          (close-pipe pipe)
+
+          ;; Older versions of Guix didn't have a 'guix repl' command, so
+          ;; emulate it.
+          (open-pipe* OPEN_BOTH "guile"
+                      "-L" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/share/guile/site/"
+                                          (effective-version))
+                      "-C" (string-append directory "/lib/guile/"
+                                          (effective-version) "/site-ccache")
+                      "-c"
+                      (object->string
+                       `(begin
+                          (primitive-load ,(search-path %load-path
+                                                        "guix/scripts/repl.scm"))
+                          ((@ (guix scripts repl) machine-repl))))))
+        pipe)))
+
+(define* (open-inferior directory #:key (command "bin/guix"))
+  "Open the inferior Guix in DIRECTORY, running 'DIRECTORY/COMMAND repl' or
+equivalent.  Return #f if the inferior could not be launched."
+  (define pipe
+    (inferior-pipe directory command))
+
+  (setvbuf pipe _IOLBF)
+  (match (read pipe)
+    (('repl-version 0 rest ...)
+     (let ((result (inferior 'pipe pipe (cons 0 rest))))
+       (inferior-eval '(use-modules (guix)) result)
+       (inferior-eval '(use-modules (gnu)) result)
+       (inferior-eval '(define %package-table (make-hash-table))
+                      result)
+       result))
+    (_
+     #f)))
+
+(define (close-inferior inferior)
+  "Close INFERIOR."
+  (close-pipe (inferior-socket inferior)))
+
+;; Non-self-quoting object of the inferior.
+(define-record-type <inferior-object>
+  (inferior-object address appearance)
+  inferior-object?
+  (address     inferior-object-address)
+  (appearance  inferior-object-appearance))
+
+(define (write-inferior-object object port)
+  (match object
+    (($ <inferior-object> _ appearance)
+     (format port "#<inferior-object ~a>" appearance))))
+
+(set-record-type-printer! <inferior-object> write-inferior-object)
+
+(define (inferior-eval exp inferior)
+  "Evaluate EXP in INFERIOR."
+  (define sexp->object
+    (match-lambda
+      (('value value)
+       value)
+      (('non-self-quoting address string)
+       (inferior-object address string))))
+
+  (write exp (inferior-socket inferior))
+  (newline (inferior-socket inferior))
+  (match (read (inferior-socket inferior))
+    (('values objects ...)
+     (apply values (map sexp->object objects)))
+    (('exception key objects ...)
+     (apply throw key (map sexp->object objects)))))
+
+\f
+;;;
+;;; Inferior packages.
+;;;
+
+(define-record-type <inferior-package>
+  (inferior-package inferior name version id)
+  inferior-package?
+  (inferior   inferior-package-inferior)
+  (name       inferior-package-name)
+  (version    inferior-package-version)
+  (id         inferior-package-id))
+
+(define (write-inferior-package package port)
+  (match package
+    (($ <inferior-package> _ name version)
+     (format port "#<inferior-package ~a@~a ~a>"
+             name version
+             (number->string (object-address package) 16)))))
+
+(set-record-type-printer! <inferior-package> write-inferior-package)
+
+(define (inferior-packages inferior)
+  "Return the list of packages known to INFERIOR."
+  (let ((result (inferior-eval
+                 '(fold-packages (lambda (package result)
+                                   (let ((id (object-address package)))
+                                     (hashv-set! %package-table id package)
+                                     (cons (list (package-name package)
+                                                 (package-version package)
+                                                 id)
+                                           result)))
+                                 '())
+                 inferior)))
+    (map (match-lambda
+           ((name version id)
+            (inferior-package inferior name version id)))
+         result)))
+
+(define (inferior-package-field package getter)
+  "Return the field of PACKAGE, an inferior package, accessed with GETTER."
+  (let ((inferior (inferior-package-inferior package))
+        (id       (inferior-package-id package)))
+    (inferior-eval `(,getter (hashv-ref %package-table ,id))
+                   inferior)))
+
+(define* (inferior-package-synopsis package #:key (translate? #t))
+  "Return the Texinfo synopsis of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-synopsis)
+                              'package-synopsis)))
+
+(define* (inferior-package-description package #:key (translate? #t))
+  "Return the Texinfo description of PACKAGE, an inferior package.  When
+TRANSLATE? is true, translate it to the current locale's language."
+  (inferior-package-field package
+                          (if translate?
+                              '(compose (@ (guix ui) P_) package-description)
+                              'package-description)))
diff --git a/tests/inferior.scm b/tests/inferior.scm
new file mode 100644
index 000000000..5e0f8ae66
--- /dev/null
+++ b/tests/inferior.scm
@@ -0,0 +1,69 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Ludovic Courtès <ludo@gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-inferior)
+  #:use-module (guix inferior)
+  #:use-module (guix packages)
+  #:use-module (gnu packages)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-64))
+
+(define %top-srcdir
+  (dirname (search-path %load-path "guix.scm")))
+
+(define %top-builddir
+  (dirname (search-path %load-compiled-path "guix.go")))
+
+\f
+(test-begin "inferior")
+
+(test-equal "open-inferior"
+  '(42 #t)
+  (let ((inferior (open-inferior %top-builddir
+                                 #:command "scripts/guix")))
+    (and (inferior? inferior)
+         (let ((a (inferior-eval '(apply * '(6 7)) inferior))
+               (b (inferior-eval '(@ (gnu packages base) coreutils)
+                                 inferior)))
+           (close-inferior inferior)
+           (list a (inferior-object? b))))))
+
+(test-equal "inferior-packages"
+  (take (sort (fold-packages (lambda (package lst)
+                               (alist-cons (package-name package)
+                                           (package-version package)
+                                           lst))
+                             '())
+              (lambda (x y)
+                (string<? (car x) (car y))))
+        10)
+  (let* ((inferior (open-inferior %top-builddir
+                                  #:command "scripts/guix"))
+         (packages (inferior-packages inferior)))
+    (and (every string? (map inferior-package-synopsis packages))
+         (begin
+           (close-inferior inferior)
+           (take (sort (map (lambda (package)
+                              (cons (inferior-package-name package)
+                                    (inferior-package-version package)))
+                            packages)
+                       (lambda (x y)
+                         (string<? (car x) (car y))))
+                 10)))))
+
+(test-end "inferior")
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 1/3] profiles: Factorize 'manifest-search-paths'.
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 1/2] Add (guix inferior) Ludovic Courtès
@ 2018-07-10 16:48   ` Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 2/3] environment: Simplify code by using manifests internally Ludovic Courtès
                     ` (3 subsequent siblings)
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/profiles.scm (manifest-search-paths): New procedure.
(profile-derivation)[builder]: Use it.
* guix/build/profiles.scm (build-etc/profile): Remove $PATH.
---
 guix/build/profiles.scm |  2 +-
 guix/profiles.scm       | 12 ++++++++++--
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/guix/build/profiles.scm b/guix/build/profiles.scm
index 819688a91..df785c85a 100644
--- a/guix/build/profiles.scm
+++ b/guix/build/profiles.scm
@@ -89,7 +89,7 @@ definitions for all the SEARCH-PATHS."
 # When GUIX_PROFILE is undefined, the various environment variables refer
 # to this specific profile generation.
 \n" port)
-      (let ((variables (evaluate-search-paths (cons $PATH search-paths)
+      (let ((variables (evaluate-search-paths search-paths
                                               (list output))))
         (for-each (write-environment-variable-definition port)
                   (map (abstract-profile output) variables))))))
diff --git a/guix/profiles.scm b/guix/profiles.scm
index e6b77e8d3..88228f155 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -91,6 +91,7 @@
             manifest-lookup
             manifest-installed?
             manifest-matching-entries
+            manifest-search-paths
 
             manifest-transaction
             manifest-transaction?
@@ -545,6 +546,14 @@ no match.."
 
   (filter matches? (manifest-entries manifest)))
 
+(define (manifest-search-paths manifest)
+  "Return the list of search path specifications that apply to MANIFEST,
+including the search path specification for $PATH."
+  (delete-duplicates
+   (cons $PATH
+         (append-map manifest-entry-search-paths
+                     (manifest-entries manifest)))))
+
 \f
 ;;;
 ;;; Manifest transactions.
@@ -1367,8 +1376,7 @@ are cross-built for TARGET."
               (map sexp->search-path-specification
                    (delete-duplicates
                     '#$(map search-path-specification->sexp
-                            (append-map manifest-entry-search-paths
-                                        (manifest-entries manifest))))))
+                            (manifest-search-paths manifest)))))
 
             (build-profile #$output '#$inputs
                            #:symlink #$(if relative-symlinks?
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 2/3] environment: Simplify code by using manifests internally.
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 1/2] Add (guix inferior) Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 1/3] profiles: Factorize 'manifest-search-paths' Ludovic Courtès
@ 2018-07-10 16:48   ` Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 2/2] pull: Use (guix inferior) to display new and upgraded packages Ludovic Courtès
                     ` (2 subsequent siblings)
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/scripts/environment.scm (strip-input-name)
(package+propagated-inputs, package-or-package+output?)
(compact): Remove.
(inputs->profile-derivation): Rename to...
(manifest->derivation): ... this.  Replace 'inputs' parameter with
'manifest'.
(input->manifest-entry): New procedure.
(package-environment-inputs): Rewrite to return a list of manifest
entries.
(options/resolve-packages): Rewrite to return a manifest.
(guix-environment): Remove 'inputs'.  Define 'paths' in terms of
'manifest-search-paths'.
---
 guix/scripts/environment.scm | 156 ++++++++++++++---------------------
 1 file changed, 61 insertions(+), 95 deletions(-)

diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index f8a9702b3..9a69e3b26 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2018 David Thompson <davet@gnu.org>
-;;; Copyright © 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015, 2016, 2017, 2018 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2018 Mike Gerwitz <mtg@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -102,35 +102,23 @@ existing environment variables with additional search paths."
                (newline)))
             (evaluate-profile-search-paths profile search-paths)))
 
-(define (strip-input-name input)
-  "Remove the name element from the tuple INPUT."
+(define (input->manifest-entry input)
+  "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a
+package."
   (match input
-    ((_ package) package)
-    ((_ package output)
-     (list package output))))
-
-(define (package+propagated-inputs package output)
-  "Return the union of PACKAGE's OUTPUT and its transitive propagated inputs."
-  (cons (list package output)
-        (map strip-input-name
-             (package-transitive-propagated-inputs package))))
-
-(define (package-or-package+output? expr)
-  "Return #t if EXPR is a package or a 2 element list consisting of a package
-and an output string."
-  (match expr
-    ((or (? package?) ; bare package object
-         ((? package?) (? string?))) ; package+output tuple
-     #t)
-    (_ #f)))
+    ((_ (? package? package))
+     (package->manifest-entry package))
+    ((_ (? package? package) output)
+     (package->manifest-entry package output))
+    (_
+     #f)))
 
 (define (package-environment-inputs package)
-  "Return a list of the transitive input packages for PACKAGE."
+  "Return a list of manifest entries corresponding to the transitive input
+packages for PACKAGE."
   ;; Remove non-package inputs such as origin records.
-  (filter package-or-package+output?
-          (map strip-input-name
-               (bag-transitive-inputs
-                (package->bag package)))))
+  (filter-map input->manifest-entry
+              (bag-transitive-inputs (package->bag package))))
 
 (define (show-help)
   (display (G_ "Usage: guix environment [OPTION]... PACKAGE... [-- COMMAND...]
@@ -287,55 +275,50 @@ COMMAND or an interactive shell in that environment.\n"))
             (_ memo)))
         '() alist))
 
-(define (compact lst)
-  "Remove all #f elements from LST."
-  (filter identity lst))
-
 (define (options/resolve-packages opts)
-  "Return OPTS with package specification strings replaced by actual
-packages."
-  (define (package->output package mode)
-    (match package
-      ((? package?)
-       (list mode package "out"))
-      (((? package? package) (? string? output))
-       (list mode package output))))
+  "Return OPTS with package specification strings replaced by manifest entries
+for the corresponding packages."
+  (define (manifest-entry=? e1 e2)
+    (and (eq? (manifest-entry-item e1) (manifest-entry-item e2))
+         (string=? (manifest-entry-output e1)
+                   (manifest-entry-output e2))))
 
   (define (packages->outputs packages mode)
     (match packages
-      ((? package-or-package+output? package) ; single package
-       (list (package->output package mode)))
-      (((? package-or-package+output?) ...) ; many packages
-       (map (cut package->output <> mode) packages))))
+      ((? package? package)
+       (if (eq? mode 'ad-hoc-package)
+           (list (package->manifest-entry package))
+           (package-environment-inputs package)))
+      (((? package? package) (? string? output))
+       (if (eq? mode 'ad-hoc-package)
+           (list (package->manifest-entry package output))
+           (package-environment-inputs package)))
+      ((lst ...)
+       (append-map (cut packages->outputs <> mode) lst))))
 
-  (define (manifest->outputs manifest)
-    (map (lambda (entry)
-           (cons 'ad-hoc-package ; manifests are implicitly ad-hoc
-                 (if (package? (manifest-entry-item entry))
-                     (list (manifest-entry-item entry)
-                           (manifest-entry-output entry))
-                     ;; Direct store paths have no output.
-                     (list (manifest-entry-item entry)))))
-         (manifest-entries manifest)))
-
-  (compact
-   (append-map (match-lambda
-                 (('package mode (? string? spec))
-                  (let-values (((package output)
-                                (specification->package+output spec)))
-                    (list (list mode package output))))
-                 (('expression mode str)
-                  ;; Add all the outputs of the package STR evaluates to.
-                  (packages->outputs (read/eval str) mode))
-                 (('load mode file)
-                  ;; Add all the outputs of the package defined in FILE.
-                  (let ((module (make-user-module '())))
-                    (packages->outputs (load* file module) mode)))
-                 (('manifest . file)
-                  (let ((module (make-user-module '((guix profiles) (gnu)))))
-                    (manifest->outputs (load* file module))))
-                 (_ '(#f)))
-               opts)))
+  (manifest
+   (delete-duplicates
+    (append-map (match-lambda
+                  (('package 'ad-hoc-package (? string? spec))
+                   (let-values (((package output)
+                                 (specification->package+output spec)))
+                     (list (package->manifest-entry package output))))
+                  (('package 'package (? string? spec))
+                   (package-environment-inputs
+                    (specification->package+output spec)))
+                  (('expression mode str)
+                   ;; Add all the outputs of the package STR evaluates to.
+                   (packages->outputs (read/eval str) mode))
+                  (('load mode file)
+                   ;; Add all the outputs of the package defined in FILE.
+                   (let ((module (make-user-module '())))
+                     (packages->outputs (load* file module) mode)))
+                  (('manifest . file)
+                   (let ((module (make-user-module '((guix profiles) (gnu)))))
+                     (manifest-entries (load* file module))))
+                  (_ '()))
+                opts)
+    manifest-entry=?)))
 
 (define* (build-environment derivations opts)
   "Build the DERIVATIONS required by the environment using the build options
@@ -350,11 +333,10 @@ in OPTS."
           (return #f)
           (built-derivations derivations)))))
 
-(define (inputs->profile-derivation inputs system bootstrap?)
-  "Return the derivation for a profile consisting of INPUTS for SYSTEM.
-BOOTSTRAP?  specifies whether to use the bootstrap Guile to build the
-profile."
-  (profile-derivation (packages->manifest inputs)
+(define (manifest->derivation manifest system bootstrap?)
+  "Return the derivation for a profile of MANIFEST.
+BOOTSTRAP? specifies whether to use the bootstrap Guile to build the profile."
+  (profile-derivation manifest
                       #:system system
 
                       ;; Packages can have conflicting inputs, or explicit
@@ -671,25 +653,9 @@ message if any test fails."
                                ;; within the container.
                                '("/bin/sh")
                                (list %default-shell))))
-           (packages   (options/resolve-packages opts))
+           (manifest   (options/resolve-packages opts))
            (mappings   (pick-all opts 'file-system-mapping))
-           (inputs     (delete-duplicates
-                        (append-map (match-lambda
-                                      (('ad-hoc-package package output)
-                                       (package+propagated-inputs package
-                                                                  output))
-                                      (('package package _)
-                                       (package-environment-inputs package)))
-                                    packages)))
-           (paths      (delete-duplicates
-                        (cons $PATH
-                              (append-map (match-lambda
-                                            ((or ((? package? p) _ ...)
-                                                 (? package? p))
-                                             (package-native-search-paths p))
-                                            (_ '()))
-                                          inputs))
-                        eq?)))
+           (paths      (manifest-search-paths manifest)))
 
       (when container? (assert-container-features))
 
@@ -714,8 +680,8 @@ message if any test fails."
             (mlet* %store-monad ((bash       (environment-bash container?
                                                                bootstrap?
                                                                system))
-                                 (prof-drv   (inputs->profile-derivation
-                                              inputs system bootstrap?))
+                                 (prof-drv   (manifest->derivation
+                                              manifest system bootstrap?))
                                  (profile -> (derivation->output-path prof-drv))
                                  (gc-root -> (assoc-ref opts 'gc-root)))
 
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 2/2] pull: Use (guix inferior) to display new and upgraded packages.
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
                     ` (2 preceding siblings ...)
  2018-07-10 16:48   ` [bug#32115] [PATCH 2/3] environment: Simplify code by using manifests internally Ludovic Courtès
@ 2018-07-10 16:48   ` Ludovic Courtès
  2018-07-10 16:48   ` [bug#32115] [PATCH 3/3] profiles: Introduce 'profile-search-paths' and use it Ludovic Courtès
  2018-07-10 16:50   ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/scripts/pull.scm (display-profile-content): Call
'display-generation'.
(display-profile-content-diff): New procedure.
(process-query)[list-generation]: Remove.
[list-generations]: New procedure.
Adjust accordingly.
* doc/guix.texi (Invoking guix pull): Update example of '-l'.
---
 doc/guix.texi         |  6 +++
 guix/scripts/pull.scm | 91 +++++++++++++++++++++++++++++++++++++------
 2 files changed, 86 insertions(+), 11 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e93b320e8..3e4bceb8a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -2786,12 +2786,18 @@ Generation 2	Jun 11 2018 11:02:49
     repository URL: https://git.savannah.gnu.org/git/guix.git
     branch: origin/master
     commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d
+  2 new packages: keepalived, libnfnetlink
+  6 packages upgraded: emacs-nix-mode@@2.0.4,
+    guile2.0-guix@@0.14.0-12.77a1aac, guix@@0.14.0-12.77a1aac,
+    heimdal@@7.5.0, milkytracker@@1.02.00, nix@@2.0.4
 
 Generation 3	Jun 13 2018 23:31:07	(current)
   guix 844cc1c
     repository URL: https://git.savannah.gnu.org/git/guix.git
     branch: origin/master
     commit: 844cc1c8f394f03b404c5bb3aee086922373490c
+  28 new packages: emacs-helm-ls-git, emacs-helm-mu, @dots{}
+  69 packages upgraded: borg@@1.1.6, cheese@@3.28.0, @dots{}
 @end example
 
 This @code{~/.config/guix/current} profile works like any other profile
diff --git a/guix/scripts/pull.scm b/guix/scripts/pull.scm
index 7202e3cc1..c61432b04 100644
--- a/guix/scripts/pull.scm
+++ b/guix/scripts/pull.scm
@@ -29,6 +29,7 @@
   #:use-module (guix gexp)
   #:use-module (guix grafts)
   #:use-module (guix monads)
+  #:autoload   (guix inferior) (open-inferior)
   #:use-module (guix scripts build)
   #:autoload   (guix self) (whole-package)
   #:autoload   (gnu packages ssh) (guile-ssh)
@@ -45,9 +46,11 @@
   #:use-module ((gnu packages certs) #:select (le-certs))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-35)
   #:use-module (srfi srfi-37)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 vlist)
   #:export (guix-pull))
 
 (module-autoload! (resolve-module '(guix scripts pull))
@@ -289,6 +292,7 @@ certificates~%"))
 (define (display-profile-content profile number)
   "Display the packages in PROFILE, generation NUMBER, in a human-readable
 way and displaying details about the channel's source code."
+  (display-generation profile number)
   (for-each (lambda (entry)
               (format #t "  ~a ~a~%"
                       (manifest-entry-name entry)
@@ -310,6 +314,68 @@ way and displaying details about the channel's source code."
              (manifest-entries
               (profile-manifest (generation-file-name profile number))))))
 
+(define (indented-string str indent)
+  "Return STR with each newline preceded by IDENT spaces."
+  (define indent-string
+    (make-list indent #\space))
+
+  (list->string
+   (string-fold-right (lambda (chr result)
+                        (if (eqv? chr #\newline)
+                            (cons chr (append indent-string result))
+                            (cons chr result)))
+                      '()
+                      str)))
+
+(define (display-profile-content-diff profile gen1 gen2)
+  "Display the changes in PROFILE GEN2 compared to generation GEN1."
+  (define (package-alist generation)
+    (fold (lambda (package lst)
+            (alist-cons (inferior-package-name package)
+                        (inferior-package-version package)
+                        lst))
+          '()
+          (let* ((directory (generation-file-name profile generation))
+                 (inferior  (open-inferior directory))
+                 (packages  (inferior-packages inferior)))
+            (close-inferior inferior)
+            packages)))
+
+  (display-profile-content profile gen2)
+  (let* ((gen1     (fold (match-lambda*
+                           (((name . version) table)
+                            (vhash-cons name version table)))
+                         vlist-null
+                         (package-alist gen1)))
+         (gen2     (package-alist gen2))
+         (new      (remove (match-lambda
+                             ((name . _)
+                              (vhash-assoc name gen1)))
+                           gen2))
+         (upgraded (filter-map (match-lambda
+                                 ((name . new-version)
+                                  (match (vhash-fold* cons '() name gen1)
+                                    (() #f)
+                                    ((= (cut sort <> version>?) old-versions)
+                                     (and (version>? new-version
+                                                     (first old-versions))
+                                          (string-append name "@"
+                                                         new-version))))))
+                               gen2)))
+    (unless (null? new)
+      (format #t (G_ "  ~h new packages: ~a~%") (length new)
+              (indented-string
+               (fill-paragraph (string-join (sort (map first new) string<?)
+                                            ", ")
+                               (- (%text-width) 4) 30)
+               4)))
+    (unless (null? upgraded)
+      (format #t (G_ "  ~h packages upgraded: ~a~%") (length upgraded)
+              (indented-string
+               (fill-paragraph (string-join (sort upgraded string<?) ", ")
+                               (- (%text-width) 4) 35)
+               4)))))
+
 (define (process-query opts)
   "Process any query specified by OPTS."
   (define profile
@@ -317,29 +383,32 @@ way and displaying details about the channel's source code."
 
   (match (assoc-ref opts 'query)
     (('list-generations pattern)
-     (define (list-generation display-function number)
-       (unless (zero? number)
-         (display-generation profile number)
-         (display-function profile number)
-         (newline)))
+     (define (list-generations profile numbers)
+       (match numbers
+         ((first rest ...)
+          (display-profile-content profile first)
+          (let loop ((numbers numbers))
+            (match numbers
+              ((first second rest ...)
+               (display-profile-content-diff profile
+                                             first second)
+               (loop (cons second rest)))
+              ((_) #t)
+              (()  #t))))))
 
      (leave-on-EPIPE
       (cond ((not (file-exists? profile))         ; XXX: race condition
              (raise (condition (&profile-not-found-error
                                 (profile profile)))))
             ((string-null? pattern)
-             (for-each (lambda (generation)
-                         (list-generation display-profile-content generation))
-                       (profile-generations profile)))
+             (list-generations profile (profile-generations profile)))
             ((matching-generations pattern profile)
              =>
              (match-lambda
                (()
                 (exit 1))
                ((numbers ...)
-                (for-each (lambda (generation)
-                            (list-generation display-profile-content generation))
-                          numbers)))))))))
+                (list-generations profile numbers)))))))))
 
 \f
 (define (guix-pull . args)
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 3/3] profiles: Introduce 'profile-search-paths' and use it.
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
                     ` (3 preceding siblings ...)
  2018-07-10 16:48   ` [bug#32115] [PATCH 2/2] pull: Use (guix inferior) to display new and upgraded packages Ludovic Courtès
@ 2018-07-10 16:48   ` Ludovic Courtès
  2018-07-10 16:50   ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:48 UTC (permalink / raw)
  To: 32115

* guix/profiles.scm (profile-search-paths): New procedure.
* guix/scripts/environment.scm (evaluate-search-paths): Remove.
(create-environment): Replace 'paths' with 'manifest'.  Use
'profile-search-paths' instead of 'evaluate-search-paths'.
(show-search-paths): Likewise.
(launch-environment): Replace 'paths' with 'manifest'.  Make 'pure?' a
keyword parameter.
(launch-environment/fork, launch-environment/container): Likewise.
(guix-environment): Remove 'paths' variable.  Adjust callers of the
above procedures accordingly.
---
 guix/profiles.scm            | 14 ++++++++++
 guix/scripts/environment.scm | 54 +++++++++++++++++-------------------
 2 files changed, 39 insertions(+), 29 deletions(-)

diff --git a/guix/profiles.scm b/guix/profiles.scm
index 88228f155..d2a794b18 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -110,6 +110,7 @@
             ca-certificate-bundle
             %default-profile-hooks
             profile-derivation
+            profile-search-paths
 
             generation-number
             generation-numbers
@@ -1400,6 +1401,19 @@ are cross-built for TARGET."
                       ;; to have no substitute to offer.
                       #:substitutable? #f)))
 
+(define* (profile-search-paths profile
+                               #:optional (manifest (profile-manifest profile))
+                               #:key (getenv (const #f)))
+  "Read the manifest of PROFILE and evaluate the values of search path
+environment variables required by PROFILE; return a list of
+specification/value pairs.  If MANIFEST is not #f, it is assumed to be the
+manifest of PROFILE, which avoids rereading it.
+
+Use GETENV to determine the current settings and report only settings not
+already effective."
+  (evaluate-search-paths (manifest-search-paths manifest)
+                         (list profile) getenv))
+
 (define (profile-regexp profile)
   "Return a regular expression that matches PROFILE's name and number."
   (make-regexp (string-append "^" (regexp-quote (basename profile))
diff --git a/guix/scripts/environment.scm b/guix/scripts/environment.scm
index 9a69e3b26..1c04800e4 100644
--- a/guix/scripts/environment.scm
+++ b/guix/scripts/environment.scm
@@ -49,11 +49,6 @@
   #:use-module (srfi srfi-98)
   #:export (guix-environment))
 
-(define (evaluate-profile-search-paths profile search-paths)
-  "Evaluate SEARCH-PATHS, a list of search-path specifications, for the
-directories in PROFILE, the store path of a profile."
-  (evaluate-search-paths search-paths (list profile)))
-
 ;; Protect some env vars from purification.  Borrowed from nix-shell.
 (define %precious-variables
   '("HOME" "USER" "LOGNAME" "DISPLAY" "TERM" "TZ" "PAGER"))
@@ -70,8 +65,8 @@ as 'HOME' and 'USER' are left untouched."
                       (((names . _) ...)
                        names)))))
 
-(define (create-environment profile paths pure?)
-  "Set the environment variables specified by PATHS for PROFILE.  When PURE?
+(define* (create-environment profile manifest #:key pure?)
+  "Set the environment variables specified by MANIFEST for PROFILE.  When PURE?
 is #t, unset the variables in the current environment.  Otherwise, augment
 existing environment variables with additional search paths."
   (when pure? (purify-environment))
@@ -84,23 +79,23 @@ existing environment variables with additional search paths."
                                  (string-append value separator current)
                                  value)
                              value)))))
-            (evaluate-profile-search-paths profile paths))
+            (profile-search-paths profile manifest))
 
   ;; Give users a way to know that they're in 'guix environment', so they can
   ;; adjust 'PS1' accordingly, for instance.  Set it to PROFILE so users can
   ;; conveniently access its contents.
   (setenv "GUIX_ENVIRONMENT" profile))
 
-(define (show-search-paths profile search-paths pure?)
-  "Display SEARCH-PATHS applied to PROFILE.  When PURE? is #t, do not augment
-existing environment variables with additional search paths."
+(define* (show-search-paths profile manifest #:key pure?)
+  "Display the search paths of MANIFEST applied to PROFILE.  When PURE? is #t,
+do not augment existing environment variables with additional search paths."
   (for-each (match-lambda
               ((search-path . value)
                (display
                 (search-path-definition search-path value
                                         #:kind (if pure? 'exact 'prefix)))
                (newline)))
-            (evaluate-profile-search-paths profile search-paths)))
+            (profile-search-paths profile manifest)))
 
 (define (input->manifest-entry input)
   "Return a manifest entry for INPUT, or #f if INPUT does not correspond to a
@@ -379,32 +374,34 @@ and suitable for 'exit'."
 (define exit/status (compose exit status->exit-code))
 (define primitive-exit/status (compose primitive-exit status->exit-code))
 
-(define (launch-environment command inputs paths pure?)
+(define* (launch-environment command profile manifest
+                             #:key pure?)
   "Run COMMAND in a new environment containing INPUTS, using the native search
 paths defined by the list PATHS.  When PURE?, pre-existing environment
 variables are cleared before setting the new ones."
   ;; Properly handle SIGINT, so pressing C-c in an interactive terminal
   ;; application works.
   (sigaction SIGINT SIG_DFL)
-  (create-environment inputs paths pure?)
+  (create-environment profile manifest #:pure? pure?)
   (match command
     ((program . args)
      (apply execlp program program args))))
 
-(define (launch-environment/fork command inputs paths pure?)
-  "Run COMMAND in a new process with an environment containing INPUTS, using
-the native search paths defined by the list PATHS.  When PURE?, pre-existing
-environment variables are cleared before setting the new ones."
+(define* (launch-environment/fork command profile manifest #:key pure?)
+  "Run COMMAND in a new process with an environment containing PROFILE, with
+the search paths specified by MANIFEST.  When PURE?, pre-existing environment
+variables are cleared before setting the new ones."
   (match (primitive-fork)
-    (0 (launch-environment command inputs paths pure?))
+    (0 (launch-environment command profile manifest
+                           #:pure? pure?))
     (pid (match (waitpid pid)
            ((_ . status) status)))))
 
 (define* (launch-environment/container #:key command bash user user-mappings
-                                       profile paths link-profile? network?)
+                                       profile manifest link-profile? network?)
   "Run COMMAND within a container that features the software in PROFILE.
-Environment variables are set according to PATHS, a list of native search
-paths.  The global shell is BASH, a file name for a GNU Bash binary in the
+Environment variables are set according to the search paths of MANIFEST.
+The global shell is BASH, a file name for a GNU Bash binary in the
 store.  When NETWORK?, access to the host system network is permitted.
 USER-MAPPINGS, a list of file system mappings, contains the user-specified
 host file systems to mount inside the container.  If USER is not #f, each
@@ -496,7 +493,7 @@ will be used for the passwd entry.  LINK-PROFILE? creates a symbolic link from
             (primitive-exit/status
              ;; A container's environment is already purified, so no need to
              ;; request it be purified again.
-             (launch-environment command profile paths #f)))
+             (launch-environment command profile manifest #:pure? #f)))
           #:namespaces (if network?
                            (delq 'net %namespaces) ; share host network
                            %namespaces)))))))
@@ -654,8 +651,7 @@ message if any test fails."
                                '("/bin/sh")
                                (list %default-shell))))
            (manifest   (options/resolve-packages opts))
-           (mappings   (pick-all opts 'file-system-mapping))
-           (paths      (manifest-search-paths manifest)))
+           (mappings   (pick-all opts 'file-system-mapping)))
 
       (when container? (assert-container-features))
 
@@ -700,7 +696,7 @@ message if any test fails."
                  ((assoc-ref opts 'dry-run?)
                   (return #t))
                  ((assoc-ref opts 'search-paths)
-                  (show-search-paths profile paths pure?)
+                  (show-search-paths profile manifest #:pure? pure?)
                   (return #t))
                  (container?
                   (let ((bash-binary
@@ -713,11 +709,11 @@ message if any test fails."
                                                   #:user user
                                                   #:user-mappings mappings
                                                   #:profile profile
-                                                  #:paths paths
+                                                  #:manifest manifest
                                                   #:link-profile? link-prof?
                                                   #:network? network?)))
                  (else
                   (return
                    (exit/status
-                    (launch-environment/fork command profile
-                                             paths pure?)))))))))))))
+                    (launch-environment/fork command profile manifest
+                                             #:pure? pure?)))))))))))))
-- 
2.18.0

^ permalink raw reply related	[flat|nested] 9+ messages in thread

* [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl).
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
                     ` (4 preceding siblings ...)
  2018-07-10 16:48   ` [bug#32115] [PATCH 3/3] profiles: Introduce 'profile-search-paths' and use it Ludovic Courtès
@ 2018-07-10 16:50   ` Ludovic Courtès
  5 siblings, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-10 16:50 UTC (permalink / raw)
  To: 32115

Oops, apologies for the mess, I sent more patches than I wanted.  I hope
you’ll find out which ones matter.  :-)

Ludo’.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* bug#32115: [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l'
  2018-07-10 16:45 [bug#32115] [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès
  2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
@ 2018-07-13 15:59 ` Ludovic Courtès
  1 sibling, 0 replies; 9+ messages in thread
From: Ludovic Courtès @ 2018-07-13 15:59 UTC (permalink / raw)
  To: 32115-done

Ludovic Courtès <ludo@gnu.org> skribis:

> This patch adds (guix inferior), a module to interact with an inferior Guix
> process, along with a ‘guix repl’ command, which spawns a REPL optionally a
> simple sexp-based protocol.
>
> ‘guix pull -l’ uses (guix inferior) to display this:
>
> Generation 1	Jun 10 2018 00:18:18
>   guix 65956ad
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: origin/master
>     commit: 65956ad3526ba09e1f7a40722c96c6ef7c0936fe
> Generation 2	Jun 11 2018 11:02:49
>   guix e0cc7f6
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: origin/master
>     commit: e0cc7f669bec22c37481dd03a7941c7d11a64f1d
>   2 new packages: keepalived, libnfnetlink
>   6 packages upgraded: emacs-nix-mode@2.0.4, guile2.0-guix@0.14.0-12.77a1aac,
>     guix@0.14.0-12.77a1aac, heimdal@7.5.0, milkytracker@1.02.00, nix@2.0.4
> Generation 3	Jun 13 2018 23:31:07
>   guix 844cc1c
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: origin/master
>     commit: 844cc1c8f394f03b404c5bb3aee086922373490c
>   28 new packages: emacs-helm-ls-git, emacs-helm-mu, emacs-helm-pass, emacs-image+,
>     emacs-package-lint, emacs-puppet-mode, emacs-wgrep-helm, go-github-com-gorilla-mux,
>     go-github-com-jonboulle-clockwork, go-github-com-spf13-pflag, libostree, ovmf-aarch64, ovmf-arm, qtwebglplugin,
>     qtwebview, r-ellipse, r-factoextra, r-factominer, r-flashclust, r-ggpubr, r-ggsci, r-ggsignif, r-hdf5r,
>     r-nbclust, r-writexl, skopeo, umoci, vkd3d
>   69 packages upgraded: borg@1.1.6, cheese@3.28.0, cpupower@4.17.1, efivar@36, emacs-orgalist@1.8,
>     emacspeak@48.0, font-gnu-unifont@11.0.01, freefall@4.17.1, glslang@3.0-3.32d3ec319, guile-bytestructures@1.0.3,
>     guile2.0-bytestructures@1.0.3, iproute2@4.17.0, ldb@1.4.0, libfilezilla@0.12.3, libndp@1.7, libraw@0.18.12,

I pushed these patches with a couple of improvements.  Let me know what
you think!

Ludo’.

^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2018-07-13 16:01 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-07-10 16:45 [bug#32115] [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès
2018-07-10 16:48 ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
2018-07-10 16:48   ` [bug#32115] [PATCH 1/2] Add (guix inferior) Ludovic Courtès
2018-07-10 16:48   ` [bug#32115] [PATCH 1/3] profiles: Factorize 'manifest-search-paths' Ludovic Courtès
2018-07-10 16:48   ` [bug#32115] [PATCH 2/3] environment: Simplify code by using manifests internally Ludovic Courtès
2018-07-10 16:48   ` [bug#32115] [PATCH 2/2] pull: Use (guix inferior) to display new and upgraded packages Ludovic Courtès
2018-07-10 16:48   ` [bug#32115] [PATCH 3/3] profiles: Introduce 'profile-search-paths' and use it Ludovic Courtès
2018-07-10 16:50   ` [bug#32115] [PATCH 1/2] Add (guix inferior) and (guix scripts repl) Ludovic Courtès
2018-07-13 15:59 ` bug#32115: [PATCH 0/2] Add (guix inferior) and improve 'guix pull -l' Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).