all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#44460] [PATCH] processes: Optionally normalize recutils output.
@ 2020-11-05  4:31 John Soo
  2020-11-05 15:49 ` [bug#44460] processes: Don't normalize Locks John Soo
                   ` (2 more replies)
  0 siblings, 3 replies; 14+ messages in thread
From: John Soo @ 2020-11-05  4:31 UTC (permalink / raw)
  To: 44460

[-- Attachment #1: Type: text/plain, Size: 1473 bytes --]

Hi Guix,

Please let me know if there is already a way to do what I want without
this patch :). I really don't know a whole lot of recutils beyond what I
learned this morning.

I was trying to extract PIDs of the child processes of sessions from the
output of guix processes.  I ran into the following problem: the PID and
the command are on the same line.

---- Begin ----
$ guix processes
SessionPID: 4278
ClientPID: 4268
ClientCommand: /gnu/store/...
ChildProcess: 4435: /gnu/store/...
ChildProcess: 4554: /gnu/store/...
ChildProcess: 4646: guile --no-auto-compile ...
---- End ----

I ended up having to use sed to remove the trailing command for the
child process.  After reading the documentation for recutils I found out
that records can also be expressed in separate record sets that can be
joined together. 

This patch adds a --normalize flag that specifies the
recutils output be put in separate record sets for Locks, Sessions, and
ChildProcesses.  For example:

---- Begin ----
PAGER=cat ./pre-inst-env guix processes --normalize
%rec: Session
%type: PID int
%type: ClientPID int
%key: PID

PID: 4278
ClientPID: 4268
ClientCommand: /gnu/store/...

%rec: Lock
%type: Session rec Session

%rec: ChildProcess
%type: Session rec Session
%type: PID int
%key: PID

Session: 4278
PID: 4435
Command: /gnu/store/...

Session: 4278
PID: 4554
Command: /gnu/store/...

Session: 4278
PID: 4646
Command: guile --no-auto-compile ...

---- End ----

What do you think?

John


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 7068 bytes --]

From 9c04e2d184977c95090ec237a6bc0334ee14a8e7 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  26 ++++++++
 guix/scripts/processes.scm | 121 +++++++++++++++++++++++++++++++++----
 2 files changed, 135 insertions(+), 12 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..ed54c26072 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12691,6 +12691,32 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ build'
+Session_PID: 4278
+PID: 4435
+
+Session_PID: 4278
+PID: 4554
+
+Session_PID: 4278
+PID: 4646
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..11cfd561b9 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -176,6 +176,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->record lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +187,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->record lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,6 +195,98 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (format-single-record port)
+  "Display denormalized session information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+                (newline port))
+            (daemon-sessions)))
+
+(define session-rec-type
+  ;; Also includes ClientCommand but it doesn't seem to be possible to express
+  ;; a plain string field (the default) without further restrictions
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID")
+
+(define lock-rec-type
+  ;; Also includes LockHeld but it doesn't seem to be possible to express
+  ;; a plain string field (the default) without further restrictions
+  "%rec: Lock
+%type: Session rec Session")
+
+(define child-process-rec-type
+  ;; Also includes Command but it doesn't seem to be possible to
+  ;; express a plain string field (the default) without further restrictions
+  "%rec: ChildProcess
+%type: Session rec Session
+%type: PID int
+%key: PID")
+
+(define (session-pid-key->field session port)
+  "Display SESSION PID as field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (format-normalized port)
+  (define sessions (daemon-sessions))
+
+  (format port session-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (session-scalars->normalized-record session port))
+   sessions)
+  (newline port)
+
+  (format port lock-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (for-each
+      (lambda (lock)
+        (session-pid-key->field session port)
+        (newline port)
+        (lock->record lock port)
+        (newline port)
+        (newline port))
+      (daemon-session-locks-held session)))
+   sessions)
+
+  (format port child-process-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (for-each
+      (lambda (process)
+        (session-pid-key->field session port)
+        (newline port)
+        (child-process->normalized-record process port)
+        (newline port)
+        (newline port))
+      (daemon-session-children session)))
+   sessions))
+
 \f
 ;;;
 ;;; Options.
@@ -205,7 +299,10 @@ active sessions, and the master 'guix-daemon' process."
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
-                  (show-version-and-exit "guix processes")))))
+                  (show-version-and-exit "guix processes")))
+        (option '("normalize") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'normalize #t result)))))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +313,13 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  --normalize            display results as normalized record sets"))
+  (newline)
   (show-bug-report-information))
 
+(define %default-options '())
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +329,12 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'normalize)
+      (#t (format-normalized port))
+      (_ (format-single-record port)))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] processes: Don't normalize Locks
  2020-11-05  4:31 [bug#44460] [PATCH] processes: Optionally normalize recutils output John Soo
@ 2020-11-05 15:49 ` John Soo
  2020-11-05 16:01 ` [bug#44460] Fixup the output of Session John Soo
  2020-11-06 23:34 ` [bug#44460] Add copyright lines John Soo
  2 siblings, 0 replies; 14+ messages in thread
From: John Soo @ 2020-11-05 15:49 UTC (permalink / raw)
  To: 44460

[-- Attachment #1: Type: text/plain, Size: 240 bytes --]

I got a little eager to normalize and put locks in their own record
set. That was unnecessary as records can have multiple of the same field
name. This new patch removes the Lock record set and puts the Lock in
the Session record.

- John


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 6612 bytes --]

From 699c66987885d91788ea0707a819270ca9fb2e1e Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  26 ++++++++++
 guix/scripts/processes.scm | 103 ++++++++++++++++++++++++++++++++-----
 2 files changed, 117 insertions(+), 12 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..ed54c26072 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12691,6 +12691,32 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ build'
+Session_PID: 4278
+PID: 4435
+
+Session_PID: 4278
+PID: 4554
+
+Session_PID: 4278
+PID: 4646
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..6828cf576e 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -176,6 +176,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->record lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +187,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->record lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,6 +195,80 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (format-single-record port)
+  "Display denormalized session information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+                (newline port))
+            (daemon-sessions)))
+
+(define session-rec-type
+  ;; Also includes ClientCommand and LockHeld but it doesn't seem to be
+  ;; possible to express a plain string field (the default) without further
+  ;; restrictions
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID")
+
+(define child-process-rec-type
+  ;; Also includes Command but it doesn't seem to be possible to
+  ;; express a plain string field (the default) without further restrictions
+  "%rec: ChildProcess
+%type: Session rec Session
+%type: PID int
+%key: PID")
+
+(define (session-key->field session port)
+  "Display SESSION PID as field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (format-normalized port)
+  (define sessions (daemon-sessions))
+
+  (format port session-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (session-scalars->normalized-record session port)
+     (for-each (lambda (lock) (lock->record lock port))
+      (daemon-session-locks-held session)))
+   sessions)
+  (newline port)
+
+  (format port child-process-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (for-each
+      (lambda (process)
+        (session-key->field session port)
+        (newline port)
+        (child-process->normalized-record process port)
+        (newline port)
+        (newline port))
+      (daemon-session-children session)))
+   sessions))
+
 \f
 ;;;
 ;;; Options.
@@ -205,7 +281,10 @@ active sessions, and the master 'guix-daemon' process."
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
-                  (show-version-and-exit "guix processes")))))
+                  (show-version-and-exit "guix processes")))
+        (option '("normalize") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'normalize #t result)))))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +295,13 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  --normalize            display results as normalized record sets"))
+  (newline)
   (show-bug-report-information))
 
+(define %default-options '())
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +311,12 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'normalize)
+      (#t (format-normalized port))
+      (_ (format-single-record port)))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Fixup the output of Session
  2020-11-05  4:31 [bug#44460] [PATCH] processes: Optionally normalize recutils output John Soo
  2020-11-05 15:49 ` [bug#44460] processes: Don't normalize Locks John Soo
@ 2020-11-05 16:01 ` John Soo
  2020-11-06 23:34 ` [bug#44460] Add copyright lines John Soo
  2 siblings, 0 replies; 14+ messages in thread
From: John Soo @ 2020-11-05 16:01 UTC (permalink / raw)
  To: 44460

[-- Attachment #1: Type: text/plain, Size: 98 bytes --]

I had too many newlines after the session recordset and too few between
session records.

- John


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 6622 bytes --]

From 26ed1a755966d20cb5f2cd8a77d55a981fe6e965 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  26 ++++++++++
 guix/scripts/processes.scm | 104 ++++++++++++++++++++++++++++++++-----
 2 files changed, 118 insertions(+), 12 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..ed54c26072 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12691,6 +12691,32 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ build'
+Session_PID: 4278
+PID: 4435
+
+Session_PID: 4278
+PID: 4554
+
+Session_PID: 4278
+PID: 4646
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..6497cd5c2f 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -176,6 +176,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->record lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +187,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->record lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,6 +195,81 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (format-single-record port)
+  "Display denormalized session information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+                (newline port))
+            (daemon-sessions)))
+
+(define session-rec-type
+  ;; Also includes ClientCommand and LockHeld but it doesn't seem to be
+  ;; possible to express a plain string field (the default) without further
+  ;; restrictions
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID")
+
+(define child-process-rec-type
+  ;; Also includes Command but it doesn't seem to be possible to
+  ;; express a plain string field (the default) without further restrictions
+  "%rec: ChildProcess
+%type: Session rec Session
+%type: PID int
+%key: PID")
+
+(define (session-key->field session port)
+  "Display SESSION PID as field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (format-normalized port)
+  (define sessions (daemon-sessions))
+
+  (format port session-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (session-scalars->normalized-record session port)
+     (for-each
+      (lambda (lock) (lock->record lock port))
+      (daemon-session-locks-held session))
+     (newline port))
+   sessions)
+
+  (format port child-process-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (for-each
+      (lambda (process)
+        (session-key->field session port)
+        (newline port)
+        (child-process->normalized-record process port)
+        (newline port)
+        (newline port))
+      (daemon-session-children session)))
+   sessions))
+
 \f
 ;;;
 ;;; Options.
@@ -205,7 +282,10 @@ active sessions, and the master 'guix-daemon' process."
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
-                  (show-version-and-exit "guix processes")))))
+                  (show-version-and-exit "guix processes")))
+        (option '("normalize") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'normalize #t result)))))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +296,13 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  --normalize            display results as normalized record sets"))
+  (newline)
   (show-bug-report-information))
 
+(define %default-options '())
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +312,12 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'normalize)
+      (#t (format-normalized port))
+      (_ (format-single-record port)))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-05  4:31 [bug#44460] [PATCH] processes: Optionally normalize recutils output John Soo
  2020-11-05 15:49 ` [bug#44460] processes: Don't normalize Locks John Soo
  2020-11-05 16:01 ` [bug#44460] Fixup the output of Session John Soo
@ 2020-11-06 23:34 ` John Soo
  2020-11-10 22:15   ` Ludovic Courtès
  2 siblings, 1 reply; 14+ messages in thread
From: John Soo @ 2020-11-06 23:34 UTC (permalink / raw)
  To: 44460

[-- Attachment #1: Type: text/plain, Size: 57 bytes --]

Sorry, forgot to add my copyright information.

- John



[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 7464 bytes --]

From 48945997d9139ddc5e288512de3dda8d5accaf44 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  27 ++++++++++
 guix/scripts/processes.scm | 105 ++++++++++++++++++++++++++++++++-----
 2 files changed, 120 insertions(+), 12 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..116706224e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12691,6 +12692,32 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ build'
+Session_PID: 4278
+PID: 4435
+
+Session_PID: 4278
+PID: 4554
+
+Session_PID: 4278
+PID: 4646
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..10fda9f223 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -176,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->record lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->record lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,6 +196,81 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (format-single-record port)
+  "Display denormalized session information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+                (newline port))
+            (daemon-sessions)))
+
+(define session-rec-type
+  ;; Also includes ClientCommand and LockHeld but it doesn't seem to be
+  ;; possible to express a plain string field (the default) without further
+  ;; restrictions
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID")
+
+(define child-process-rec-type
+  ;; Also includes Command but it doesn't seem to be possible to
+  ;; express a plain string field (the default) without further restrictions
+  "%rec: ChildProcess
+%type: Session rec Session
+%type: PID int
+%key: PID")
+
+(define (session-key->field session port)
+  "Display SESSION PID as field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (format-normalized port)
+  (define sessions (daemon-sessions))
+
+  (format port session-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (session-scalars->normalized-record session port)
+     (for-each
+      (lambda (lock) (lock->record lock port))
+      (daemon-session-locks-held session))
+     (newline port))
+   sessions)
+
+  (format port child-process-rec-type)
+  (newline port)
+  (newline port)
+  (for-each
+   (lambda (session)
+     (for-each
+      (lambda (process)
+        (session-key->field session port)
+        (newline port)
+        (child-process->normalized-record process port)
+        (newline port)
+        (newline port))
+      (daemon-session-children session)))
+   sessions))
+
 \f
 ;;;
 ;;; Options.
@@ -205,7 +283,10 @@ active sessions, and the master 'guix-daemon' process."
                   (exit 0)))
         (option '(#\V "version") #f #f
                 (lambda args
-                  (show-version-and-exit "guix processes")))))
+                  (show-version-and-exit "guix processes")))
+        (option '("normalize") #f #f
+                (lambda (opt name arg result)
+                  (alist-cons 'normalize #t result)))))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +297,13 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  --normalize            display results as normalized record sets"))
+  (newline)
   (show-bug-report-information))
 
+(define %default-options '())
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +313,12 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'normalize)
+      (#t (format-normalized port))
+      (_ (format-single-record port)))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-06 23:34 ` [bug#44460] Add copyright lines John Soo
@ 2020-11-10 22:15   ` Ludovic Courtès
  2020-11-11 17:51     ` John Soo
  0 siblings, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2020-11-10 22:15 UTC (permalink / raw)
  To: John Soo; +Cc: 44460

Hi John,

John Soo <jsoo1@asu.edu> skribis:

>>From 48945997d9139ddc5e288512de3dda8d5accaf44 Mon Sep 17 00:00:00 2001
> From: John Soo <jsoo1@asu.edu>
> Date: Wed, 4 Nov 2020 07:51:52 -0800
> Subject: [PATCH] processes: Optionally normalize recutils output.
>
> * guix/scripts/processes.scm: Add "normalize" flag

[...]

> +@table @code
> +@item --normalize
> +Normalize the output records into record sets (@pxref{Record Sets,,,
> +recutils, GNU recutils manual}).  Normalizing into record sets allows
> +joins across record types.
> +
> +@example
> +$ guix processes --normalize | \
> +    recsel \
> +    -j Session \
> +    -t ChildProcess \
> +    -p Session.PID,PID \
> +    -e 'Session.ClientCommand ~ build'
> +Session_PID: 4278
> +PID: 4435
> +
> +Session_PID: 4278
> +PID: 4554
> +
> +Session_PID: 4278
> +PID: 4646
> +@end example
> +@end table

Nice!  Right above the example, I’d suggest adding a sentence like “The
example below lists…” (what does it list actually? :-)).

In the default format, I wonder if we could already change split
‘ChildProcess’ into ‘ChildPID’ and ‘ChildCommand’, as you had initially
proposed on IRC; would that work?

> +(define (lock->record lock port)
> +  (format port "LockHeld: ~a~%" lock))

Maybe ‘lock->recutils’ for consistency and to avoid confusion with
Scheme “records”?

> +(define (format-single-record port)

Maybe ‘daemon-sessions->recutils’?  Should ‘sessions’ be a parameter for
clarity?

> +  "Display denormalized session information to PORT."
> +  (for-each (lambda (session)
> +              (daemon-session->recutils session port)
> +                (newline port))
                  ^
Indentation is off.

> +(define (child-process->normalized-record process port)
> +  "Display PROCESS record on PORT in normalized form"
> +  (format port "PID: ~a" (process-id process))
> +  (newline port)
> +  (format port "Command:~{ ~a~}" (process-command process)))
> +
> +(define (format-normalized port)

Please add a docstring.  Perhaps make ‘sessions’ a parameter?

> +  (define sessions (daemon-sessions))
> +
> +  (format port session-rec-type)

As reported by ‘-Wformat’, passing a non-literal format string is risky;
write this instead:

  (display session-rec-type port)

> +  (newline port)
> +  (newline port)
> +  (for-each
> +   (lambda (session)

Preferable indent ‘for-each’ like so:

  (for-each (lambda (session)

Likewise below.

> +  (display (G_ "
> +  --normalize            display results as normalized record sets"))

Should it be ‘--format=normalized’ (just like we have ‘--format’ in
‘guix describe’, for instance)?

Could you send an updated patch?

Thanks!

Ludo’.




^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-10 22:15   ` Ludovic Courtès
@ 2020-11-11 17:51     ` John Soo
  2020-11-11 17:59       ` John Soo
  2020-11-12 10:58       ` Ludovic Courtès
  0 siblings, 2 replies; 14+ messages in thread
From: John Soo @ 2020-11-11 17:51 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

[-- Attachment #1: Type: text/plain, Size: 3553 bytes --]

Hello Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

>> +@table @code
>> +@item --normalize
>> +Normalize the output records into record sets (@pxref{Record Sets,,,
>> +recutils, GNU recutils manual}).  Normalizing into record sets allows
>> +joins across record types.
>> +
>> +@example
>> +$ guix processes --normalize | \
>> +    recsel \
>> +    -j Session \
>> +    -t ChildProcess \
>> +    -p Session.PID,PID \
>> +    -e 'Session.ClientCommand ~ build'
>> +Session_PID: 4278
>> +PID: 4435
>> +
>> +Session_PID: 4278
>> +PID: 4554
>> +
>> +Session_PID: 4278
>> +PID: 4646
>> +@end example
>> +@end table
>
> Nice!  Right above the example, I’d suggest adding a sentence like “The
> example below lists…” (what does it list actually? :-)).
>
> In the default format, I wonder if we could already change split
> ‘ChildProcess’ into ‘ChildPID’ and ‘ChildCommand’, as you had initially
> proposed on IRC; would that work?

I think we could do that, but I had two reasons to use the normalized
format instead.

* Backwards incompatibility - I didn't want to break any existing scripts
  that may exist.

* Still not normalized - how can I search for just the child processes
  associated with a particular command?

I wouldn't be opposed to splitting ChildProcess into ChildPID and
ChildCommand.  I would like it best if that change was made in addition
to adding the normalized version, since the normalized version allows
more functionality.

>> +(define (lock->record lock port)
>> +  (format port "LockHeld: ~a~%" lock))
>
> Maybe ‘lock->recutils’ for consistency and to avoid confusion with
> Scheme “records”?

Done.

>> +(define (format-single-record port)
>
> Maybe ‘daemon-sessions->recutils’?  Should ‘sessions’ be a parameter for
> clarity?

Much better, thank you. I updated the normalized version too.

>
>> +  "Display denormalized session information to PORT."
>> +  (for-each (lambda (session)
>> +              (daemon-session->recutils session port)
>> +                (newline port))
>                   ^
> Indentation is off.

Fixed.

>> +(define (child-process->normalized-record process port)
>> +  "Display PROCESS record on PORT in normalized form"
>> +  (format port "PID: ~a" (process-id process))
>> +  (newline port)
>> +  (format port "Command:~{ ~a~}" (process-command process)))
>> +
>> +(define (format-normalized port)
>
> Please add a docstring.  Perhaps make ‘sessions’ a parameter?

Done.

>> +  (define sessions (daemon-sessions))
>> +
>> +  (format port session-rec-type)
>
> As reported by ‘-Wformat’, passing a non-literal format string is risky;
> write this instead:
>
>   (display session-rec-type port)

Done.

>> +  (newline port)
>> +  (newline port)
>> +  (for-each
>> +   (lambda (session)
>
> Preferable indent ‘for-each’ like so:
>
>   (for-each (lambda (session)

Done.

> Likewise below.
>
>> +  (display (G_ "
>> +  --normalize            display results as normalized record sets"))
>
> Should it be ‘--format=normalized’ (just like we have ‘--format’ in
> ‘guix describe’, for instance)?

That makes sense. What do you think of

Other changes:

* Updated the record descriptors to include the possible
  fields for sessions. I got some nice guidance from the recutils irc on
  that.

* Put the PID and Command first for the ChildProcess

* Add a --list-formats like guix describe has.

Thanks!

John


[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 8681 bytes --]

From a19828e504288c03c7e17a88ac2a35eb70ab302c Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  29 ++++++++
 guix/scripts/processes.scm | 132 +++++++++++++++++++++++++++++++------
 2 files changed, 142 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..5df6096f39 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12691,6 +12692,34 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.  The example below lists the PID of each
+ChildProcess and the associated PID for Session that spawned the
+ChildProcess where the Session was started using guix build.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ "guix build'"
+PID: 4435
+Session_PID: 4278
+
+PID: 4554
+Session_PID: 4278
+
+PID: 4646
+Session_PID: 4278
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..af1d220774 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -176,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->recutils lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->recutils lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,19 +196,89 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (daemon-sessions->recutils port sessions)
+  "Display denormalized SESSIONS information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+              (newline port))
+            sessions))
+
+(define session-rec-type
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID
+%mandatory: ClientPID ClientCommand
+%allowed: LockHeld")
+
+(define child-process-rec-type
+  "%rec: ChildProcess
+%type: PID int
+%type: Session rec Session
+%key: PID
+%mandatory: Command")
+
+(define (session-key->recutils session port)
+  "Display SESSION PID as a recutils field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (daemon-sessions->normalized-record port sessions)
+  "Display SESSIONS recutils on PORT in normalized form"
+  (display session-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (session-scalars->normalized-record session port)
+              (for-each (lambda (lock)
+                          (lock->recutils lock port))
+                        (daemon-session-locks-held session))
+              (newline port))
+            sessions)
+
+  (display child-process-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (process)
+                          (child-process->normalized-record process port)
+                          (newline port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-children session)))
+            sessions))
+
 \f
 ;;;
 ;;; Options.
 ;;;
 
-(define %options
-  (list (option '(#\h "help") #f #f
-                (lambda args
-                  (show-help)
-                  (exit 0)))
-        (option '(#\V "version") #f #f
-                (lambda args
-                  (show-version-and-exit "guix processes")))))
+(define %available-formats
+  '("recutils" "recutils-normalized"))
+
+(define (list-formats)
+  (display (G_ "The available formats are:\n"))
+  (newline)
+  (for-each (lambda (f)
+              (format #t "  - ~a~%" f))
+            %available-formats))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +289,33 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  -f, --format=FORMAT    display results as normalized record sets"))
+  (newline)
+  (display (G_ "
+  --list-formats         display available formats"))
   (show-bug-report-information))
 
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix processes")))
+        (option '(#\f "format") #t #f
+                (lambda (opt name arg result)
+                  (unless (member arg %available-formats)
+                    (leave (G_ "~a: unsupported output format~%") arg))
+                  (alist-cons 'format (string->symbol arg) result)))
+        (option '("list-formats") #f #f
+                (lambda (opt name arg result)
+                  (list-formats)
+                  (exit 0)))))
+
+(define %default-options '((format . recutils)))
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +325,13 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'format)
+      ('recutils-normalized
+       (daemon-sessions->normalized-record port (daemon-sessions)))
+      (_ (daemon-sessions->recutils port (daemon-sessions))))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-11 17:51     ` John Soo
@ 2020-11-11 17:59       ` John Soo
  2020-11-11 18:47         ` John Soo
  2020-11-12 10:58       ` Ludovic Courtès
  1 sibling, 1 reply; 14+ messages in thread
From: John Soo @ 2020-11-11 17:59 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

[-- Attachment #1: Type: text/plain, Size: 91 bytes --]

I realized I slightly altered the --help format. That is fixed here.

Thanks again,

John


[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 8681 bytes --]

From 7d03d98ef9e2164723db09c83b1c6973b2ad916e Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "normalize" flag
---
 doc/guix.texi              |  29 ++++++++
 guix/scripts/processes.scm | 132 +++++++++++++++++++++++++++++++------
 2 files changed, 142 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..5df6096f39 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12691,6 +12692,34 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --normalize
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.  The example below lists the PID of each
+ChildProcess and the associated PID for Session that spawned the
+ChildProcess where the Session was started using guix build.
+
+@example
+$ guix processes --normalize | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ "guix build'"
+PID: 4435
+Session_PID: 4278
+
+PID: 4554
+Session_PID: 4278
+
+PID: 4646
+Session_PID: 4278
+@end example
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..02515b1c1d 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -176,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->recutils lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->recutils lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,19 +196,89 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (daemon-sessions->recutils port sessions)
+  "Display denormalized SESSIONS information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+              (newline port))
+            sessions))
+
+(define session-rec-type
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID
+%mandatory: ClientPID ClientCommand
+%allowed: LockHeld")
+
+(define child-process-rec-type
+  "%rec: ChildProcess
+%type: PID int
+%type: Session rec Session
+%key: PID
+%mandatory: Command")
+
+(define (session-key->recutils session port)
+  "Display SESSION PID as a recutils field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (daemon-sessions->normalized-record port sessions)
+  "Display SESSIONS recutils on PORT in normalized form"
+  (display session-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (session-scalars->normalized-record session port)
+              (for-each (lambda (lock)
+                          (lock->recutils lock port))
+                        (daemon-session-locks-held session))
+              (newline port))
+            sessions)
+
+  (display child-process-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (process)
+                          (child-process->normalized-record process port)
+                          (newline port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-children session)))
+            sessions))
+
 \f
 ;;;
 ;;; Options.
 ;;;
 
-(define %options
-  (list (option '(#\h "help") #f #f
-                (lambda args
-                  (show-help)
-                  (exit 0)))
-        (option '(#\V "version") #f #f
-                (lambda args
-                  (show-version-and-exit "guix processes")))))
+(define %available-formats
+  '("recutils" "recutils-normalized"))
+
+(define (list-formats)
+  (display (G_ "The available formats are:\n"))
+  (newline)
+  (for-each (lambda (f)
+              (format #t "  - ~a~%" f))
+            %available-formats))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +289,33 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  -f, --format=FORMAT    display results as normalized record sets"))
+  (display (G_ "
+  --list-formats         display available formats"))
+  (newline)
   (show-bug-report-information))
 
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix processes")))
+        (option '(#\f "format") #t #f
+                (lambda (opt name arg result)
+                  (unless (member arg %available-formats)
+                    (leave (G_ "~a: unsupported output format~%") arg))
+                  (alist-cons 'format (string->symbol arg) result)))
+        (option '("list-formats") #f #f
+                (lambda (opt name arg result)
+                  (list-formats)
+                  (exit 0)))))
+
+(define %default-options '((format . recutils)))
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +325,13 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'format)
+      ('recutils-normalized
+       (daemon-sessions->normalized-record port (daemon-sessions)))
+      (_ (daemon-sessions->recutils port (daemon-sessions))))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-11 17:59       ` John Soo
@ 2020-11-11 18:47         ` John Soo
  0 siblings, 0 replies; 14+ messages in thread
From: John Soo @ 2020-11-11 18:47 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

[-- Attachment #1: Type: text/plain, Size: 109 bytes --]

Oops! I also forget to change the flag names in the docs and the commit
message. Fixing those here.

- John


[-- Attachment #2: 0001-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 8999 bytes --]

From 6b3bdfcee7644b36f4e70d9f2529af11e7098333 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "format" and "list-formats" flag
---
 doc/guix.texi              |  39 +++++++++++
 guix/scripts/processes.scm | 132 +++++++++++++++++++++++++++++++------
 2 files changed, 152 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..5dc51a1111 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12691,6 +12692,44 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --format=@var{format}
+@itemx -f @var{format}
+Produce output in the specified @var{format}, one of:
+
+@table @code
+@item recutils
+The default option. It outputs a set of Session recutils records
+that include each ChildProcess as a field.
+
+@item recutils-normalized
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.  The example below lists the PID of each
+ChildProcess and the associated PID for Session that spawned the
+ChildProcess where the Session was started using guix build.
+
+@example
+$ guix processes --format=recutils-normalized | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ "guix build'"
+PID: 4435
+Session_PID: 4278
+
+PID: 4554
+Session_PID: 4278
+
+PID: 4646
+Session_PID: 4278
+@end example
+@end table
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..02515b1c1d 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -176,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->recutils lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -184,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->recutils lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildProcess: ~a:~{ ~a~}~%"
@@ -193,19 +196,89 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (daemon-sessions->recutils port sessions)
+  "Display denormalized SESSIONS information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+              (newline port))
+            sessions))
+
+(define session-rec-type
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID
+%mandatory: ClientPID ClientCommand
+%allowed: LockHeld")
+
+(define child-process-rec-type
+  "%rec: ChildProcess
+%type: PID int
+%type: Session rec Session
+%key: PID
+%mandatory: Command")
+
+(define (session-key->recutils session port)
+  "Display SESSION PID as a recutils field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (daemon-sessions->normalized-record port sessions)
+  "Display SESSIONS recutils on PORT in normalized form"
+  (display session-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (session-scalars->normalized-record session port)
+              (for-each (lambda (lock)
+                          (lock->recutils lock port))
+                        (daemon-session-locks-held session))
+              (newline port))
+            sessions)
+
+  (display child-process-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (process)
+                          (child-process->normalized-record process port)
+                          (newline port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-children session)))
+            sessions))
+
 \f
 ;;;
 ;;; Options.
 ;;;
 
-(define %options
-  (list (option '(#\h "help") #f #f
-                (lambda args
-                  (show-help)
-                  (exit 0)))
-        (option '(#\V "version") #f #f
-                (lambda args
-                  (show-version-and-exit "guix processes")))))
+(define %available-formats
+  '("recutils" "recutils-normalized"))
+
+(define (list-formats)
+  (display (G_ "The available formats are:\n"))
+  (newline)
+  (for-each (lambda (f)
+              (format #t "  - ~a~%" f))
+            %available-formats))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -216,8 +289,33 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  -f, --format=FORMAT    display results as normalized record sets"))
+  (display (G_ "
+  --list-formats         display available formats"))
+  (newline)
   (show-bug-report-information))
 
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix processes")))
+        (option '(#\f "format") #t #f
+                (lambda (opt name arg result)
+                  (unless (member arg %available-formats)
+                    (leave (G_ "~a: unsupported output format~%") arg))
+                  (alist-cons 'format (string->symbol arg) result)))
+        (option '("list-formats") #f #f
+                (lambda (opt name arg result)
+                  (list-formats)
+                  (exit 0)))))
+
+(define %default-options '((format . recutils)))
+
 \f
 ;;;
 ;;; Entry point.
@@ -227,17 +325,13 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'format)
+      ('recutils-normalized
+       (daemon-sessions->normalized-record port (daemon-sessions)))
+      (_ (daemon-sessions->recutils port (daemon-sessions))))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-11 17:51     ` John Soo
  2020-11-11 17:59       ` John Soo
@ 2020-11-12 10:58       ` Ludovic Courtès
  2020-11-12 15:37         ` John Soo
  1 sibling, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2020-11-12 10:58 UTC (permalink / raw)
  To: John Soo; +Cc: 44460

Hi John,

John Soo <jsoo1@asu.edu> skribis:


[...]

>>> +Session_PID: 4278
>>> +PID: 4646
>>> +@end example
>>> +@end table
>>
>> Nice!  Right above the example, I’d suggest adding a sentence like “The
>> example below lists…” (what does it list actually? :-)).
>>
>> In the default format, I wonder if we could already change split
>> ‘ChildProcess’ into ‘ChildPID’ and ‘ChildCommand’, as you had initially
>> proposed on IRC; would that work?
>
> I think we could do that, but I had two reasons to use the normalized
> format instead.
>
> * Backwards incompatibility - I didn't want to break any existing scripts
>   that may exist.
>
> * Still not normalized - how can I search for just the child processes
>   associated with a particular command?

Like:

  guix processes | recsel -e 'ClientCommand ~ "xyz"' -p ChildProcess

?

Actually what does “normalized” mean in this context?

> I wouldn't be opposed to splitting ChildProcess into ChildPID and
> ChildCommand.  I would like it best if that change was made in addition
> to adding the normalized version, since the normalized version allows
> more functionality.

I would think it’s OK to break compatibility on just these
“ChildProcess” fields.

> * Updated the record descriptors to include the possible
>   fields for sessions. I got some nice guidance from the recutils irc on
>   that.
>
> * Put the PID and Command first for the ChildProcess
>
> * Add a --list-formats like guix describe has.

Great, I’ll take a look.  Thanks!

Ludo’.




^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-12 10:58       ` Ludovic Courtès
@ 2020-11-12 15:37         ` John Soo
  2020-11-12 20:29           ` Ludovic Courtès
  0 siblings, 1 reply; 14+ messages in thread
From: John Soo @ 2020-11-12 15:37 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

>> * Still not normalized - how can I search for just the child processes
>>   associated with a particular command?
>
> Like:
>
>   guix processes | recsel -e 'ClientCommand ~ "xyz"' -p ChildProcess
>
> ?
>
> Actually what does “normalized” mean in this context?

Excellent question. I was thinking along the lines of database
normalization. The default output has multi-valued fields for child
processes, so the idea is to make them their own record set.  Does that
make sense?

An aside - Probably to be entirely honest about normalizing the output,
locks really would be in a separate record set too.

Another challenge is making sure the user can understand what
"normalized" means.  I am not sure readers of the manual/cli help will
be able to infer what it means from context.  On the other hand, it is
such a small use case that it seems imbalanced to provide a lot of
background for the term "normal". What do you think?

>> I wouldn't be opposed to splitting ChildProcess into ChildPID and
>> ChildCommand.  I would like it best if that change was made in addition
>> to adding the normalized version, since the normalized version allows
>> more functionality.
>
> I would think it’s OK to break compatibility on just these
> “ChildProcess” fields.

Ok. Would it be ok if I put that in a separate commit?

Thanks again!

- John




^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-12 15:37         ` John Soo
@ 2020-11-12 20:29           ` Ludovic Courtès
  2020-11-13  5:33             ` John Soo
  0 siblings, 1 reply; 14+ messages in thread
From: Ludovic Courtès @ 2020-11-12 20:29 UTC (permalink / raw)
  To: John Soo; +Cc: 44460

Hi,

John Soo <jsoo1@asu.edu> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> * Still not normalized - how can I search for just the child processes
>>>   associated with a particular command?
>>
>> Like:
>>
>>   guix processes | recsel -e 'ClientCommand ~ "xyz"' -p ChildProcess
>>
>> ?
>>
>> Actually what does “normalized” mean in this context?
>
> Excellent question. I was thinking along the lines of database
> normalization. The default output has multi-valued fields for child
> processes, so the idea is to make them their own record set.  Does that
> make sense?

Yes it does!  Initially I wondered if it was a term used in recutils,
but apparently it’s not.

> An aside - Probably to be entirely honest about normalizing the output,
> locks really would be in a separate record set too.

Yeah.

> Another challenge is making sure the user can understand what
> "normalized" means.  I am not sure readers of the manual/cli help will
> be able to infer what it means from context.  On the other hand, it is
> such a small use case that it seems imbalanced to provide a lot of
> background for the term "normal". What do you think?

Sure.

Thinking more about it, to me the appeal of recutils is that it’s both
human- and machine-readable.  But here we end up having a specific
machine-readable variant.  But yeah, maybe that’s unavoidable.

>>> I wouldn't be opposed to splitting ChildProcess into ChildPID and
>>> ChildCommand.  I would like it best if that change was made in addition
>>> to adding the normalized version, since the normalized version allows
>>> more functionality.
>>
>> I would think it’s OK to break compatibility on just these
>> “ChildProcess” fields.
>
> Ok. Would it be ok if I put that in a separate commit?

Yes (you mean in addition to ‘-f normalized’, right?).

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-12 20:29           ` Ludovic Courtès
@ 2020-11-13  5:33             ` John Soo
  2020-11-13 16:16               ` John Soo
  0 siblings, 1 reply; 14+ messages in thread
From: John Soo @ 2020-11-13  5:33 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

[-- Attachment #1: Type: text/plain, Size: 2507 bytes --]

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> Yes it does!  Initially I wondered if it was a term used in recutils,
> but apparently it’s not.

Do you think it is the best term to use?

>> An aside - Probably to be entirely honest about normalizing the output,
>> locks really would be in a separate record set too.
>
> Yeah.

I made Lock a separate record set. This exposes a limitation with
recutils. It is not possible to join more than two record sets together
(though they mentioned it as a goal on IRC).

>> Another challenge is making sure the user can understand what
>> "normalized" means.  I am not sure readers of the manual/cli help will
>> be able to infer what it means from context.  On the other hand, it is
>> such a small use case that it seems imbalanced to provide a lot of
>> background for the term "normal". What do you think?
>
> Sure.
>
> Thinking more about it, to me the appeal of recutils is that it’s both
> human- and machine-readable.  But here we end up having a specific
> machine-readable variant.  But yeah, maybe that’s unavoidable.

I suppose the normalized version is a little less human-readable.  It
would be behind a flag, though.  Is that a reasonable compromise?

After some thought I realize that the normalized version isn't that much
more useful than the default, but it does enable things like:

---- Example ----
$ guix processes -f normalized \
  | recsel \
    -t ChildProcess \
    -j Session \
    -p PID,Session.PID \
  | recfmt '{{PID}} {{Session.PID}}'
23607 2356724713 2356725002 23576
---- Example ----

This will format all the (PID, Session) pairs. Whereas the
non-normalized version would only print one PID given the following.

---- Example ----
$ guix processes \
  | recsel -p ChildPID,SessionPID \
  | recfmt '{{ChildPID}} {{SessionPID}}'
23607 23567
---

>>>> I wouldn't be opposed to splitting ChildProcess into ChildPID and
>>>> ChildCommand.  I would like it best if that change was made in addition
>>>> to adding the normalized version, since the normalized version allows
>>>> more functionality.
>>>
>>> I would think it’s OK to break compatibility on just these
>>> “ChildProcess” fields.
>>
>> Ok. Would it be ok if I put that in a separate commit?
>
> Yes (you mean in addition to ‘-f normalized’, right?).

I changed ChildProcess: pid: command to ChildPID and ChildCommand in the
default in a separate commit.

Thanks again,

John


[-- Attachment #2: 0001-processes-Put-ChildProcess-and-ChildPID-on-separate-.patch --]
[-- Type: text/x-patch, Size: 4024 bytes --]

From 6082c559d1200e632b3fb45eb0633d28829667a1 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Thu, 12 Nov 2020 21:16:48 -0800
Subject: [PATCH 1/2] processes: Put ChildProcess and ChildPID on separate
 lines.

* guix/scripts/processes.scm: Put child process information in
separate fields.
* doc/guix.texi: Document change in output of guix processes.
---
 doc/guix.texi              | 22 +++++++++++++---------
 guix/scripts/processes.scm |  6 ++++--
 2 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..e8814c686c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12661,9 +12662,12 @@ ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock
 LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock
 LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock
-ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800
-ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800
-ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800
+ChildPID: 20495
+ChildCommand: guix offload x86_64-linux 7200 1 28800
+ChildPID: 27733
+ChildCommand: guix offload x86_64-linux 7200 1 28800
+ChildPID: 27793
+ChildCommand: guix offload x86_64-linux 7200 1 28800
 @end example
 
 In this example we see that @command{guix-daemon} has three clients:
@@ -12672,12 +12676,12 @@ integration tool; their process identifier (PID) is given by the
 @code{ClientPID} field.  The @code{SessionPID} field gives the PID of the
 @command{guix-daemon} sub-process of this particular session.
 
-The @code{LockHeld} fields show which store items are currently locked by this
-session, which corresponds to store items being built or substituted (the
-@code{LockHeld} field is not displayed when @command{guix processes} is not
-running as root).  Last, by looking at the @code{ChildProcess} field, we
-understand that these three builds are being offloaded (@pxref{Daemon Offload
-Setup}).
+The @code{LockHeld} fields show which store items are currently locked
+by this session, which corresponds to store items being built or
+substituted (the @code{LockHeld} field is not displayed when
+@command{guix processes} is not running as root).  Last, by looking at
+the @code{ChildPID} and @code{ChildCommand} fields, we understand that
+these three builds are being offloaded (@pxref{Daemon Offload Setup}).
 
 The output is in Recutils format so we can use the handy @command{recsel}
 command to select sessions of interest (@pxref{Selection Expressions,,,
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..3a7ea0b89c 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -188,8 +189,9 @@ active sessions, and the master 'guix-daemon' process."
               (format port "LockHeld: ~a~%" lock))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
-              (format port "ChildProcess: ~a:~{ ~a~}~%"
-                      (process-id process)
+              (format port "ChildPID: ~a~%"
+                      (process-id process))
+              (format port "ChildCommand: :~{ ~a~}~%"
                       (process-command process)))
             (daemon-session-children session)))
 
-- 
2.29.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 8490 bytes --]

From 81cdc826a89b4f5ee480faa620f3e8f3e12d8979 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH 2/2] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "format" and "list-formats" flag.
* doc/guix.texi: Document new flags.
---
 doc/guix.texi              |  38 ++++++++++
 guix/scripts/processes.scm | 144 ++++++++++++++++++++++++++++++++-----
 2 files changed, 163 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e8814c686c..7da24977cc 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12695,6 +12695,44 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --format=@var{format}
+@itemx -f @var{format}
+Produce output in the specified @var{format}, one of:
+
+@table @code
+@item recutils
+The default option. It outputs a set of Session recutils records
+that include each ChildProcess as a field.
+
+@item recutils-normalized
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.  The example below lists the PID of each
+ChildProcess and the associated PID for Session that spawned the
+ChildProcess where the Session was started using guix build.
+
+@example
+$ guix processes --format=recutils-normalized | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ "guix build'"
+PID: 4435
+Session_PID: 4278
+
+PID: 4554
+Session_PID: 4278
+
+PID: 4646
+Session_PID: 4278
+@end example
+@end table
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index 3a7ea0b89c..bcc541badb 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -177,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->recutils lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -185,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->recutils lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildPID: ~a~%"
@@ -195,19 +197,102 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (daemon-sessions->recutils port sessions)
+  "Display denormalized SESSIONS information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+              (newline port))
+            sessions))
+
+(define session-rec-type
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID
+%mandatory: ClientPID ClientCommand")
+
+(define lock-rec-type
+  "%rec: Lock
+%mandatory: LockHeld
+%type: Session rec Session")
+
+(define child-process-rec-type
+  "%rec: ChildProcess
+%type: PID int
+%type: Session rec Session
+%key: PID
+%mandatory: Command")
+
+(define (session-key->recutils session port)
+  "Display SESSION PID as a recutils field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (daemon-sessions->normalized-record port sessions)
+  "Display SESSIONS recutils on PORT in normalized form"
+  (display session-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (session-scalars->normalized-record session port)
+              (newline port))
+            sessions)
+
+  (display lock-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (lock)
+                          (lock->recutils "testing testing" port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-locks-held session)))
+            sessions)
+
+  (display child-process-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (process)
+                          (child-process->normalized-record process port)
+                          (newline port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-children session)))
+            sessions))
+
 \f
 ;;;
 ;;; Options.
 ;;;
 
-(define %options
-  (list (option '(#\h "help") #f #f
-                (lambda args
-                  (show-help)
-                  (exit 0)))
-        (option '(#\V "version") #f #f
-                (lambda args
-                  (show-version-and-exit "guix processes")))))
+(define %available-formats
+  '("recutils" "normalized"))
+
+(define (list-formats)
+  (display (G_ "The available formats are:\n"))
+  (newline)
+  (for-each (lambda (f)
+              (format #t "  - ~a~%" f))
+            %available-formats))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -218,8 +303,33 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  -f, --format=FORMAT    display results as normalized record sets"))
+  (display (G_ "
+  --list-formats         display available formats"))
+  (newline)
   (show-bug-report-information))
 
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix processes")))
+        (option '(#\f "format") #t #f
+                (lambda (opt name arg result)
+                  (unless (member arg %available-formats)
+                    (leave (G_ "~a: unsupported output format~%") arg))
+                  (alist-cons 'format (string->symbol arg) result)))
+        (option '("list-formats") #f #f
+                (lambda (opt name arg result)
+                  (list-formats)
+                  (exit 0)))))
+
+(define %default-options '((format . recutils)))
+
 \f
 ;;;
 ;;; Entry point.
@@ -229,17 +339,13 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'format)
+      ('normalized
+       (daemon-sessions->normalized-record port (daemon-sessions)))
+      (_ (daemon-sessions->recutils port (daemon-sessions))))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* [bug#44460] Add copyright lines
  2020-11-13  5:33             ` John Soo
@ 2020-11-13 16:16               ` John Soo
  2020-11-29 22:49                 ` bug#44460: " Ludovic Courtès
  0 siblings, 1 reply; 14+ messages in thread
From: John Soo @ 2020-11-13 16:16 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 44460

[-- Attachment #1: Type: text/plain, Size: 86 bytes --]

Hello again,

No big changes here, I just forgot to update the manual again.

- John


[-- Attachment #2: 0001-processes-Put-ChildProcess-and-ChildPID-on-separate-.patch --]
[-- Type: text/x-patch, Size: 4024 bytes --]

From 6082c559d1200e632b3fb45eb0633d28829667a1 Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Thu, 12 Nov 2020 21:16:48 -0800
Subject: [PATCH 1/2] processes: Put ChildProcess and ChildPID on separate
 lines.

* guix/scripts/processes.scm: Put child process information in
separate fields.
* doc/guix.texi: Document change in output of guix processes.
---
 doc/guix.texi              | 22 +++++++++++++---------
 guix/scripts/processes.scm |  6 ++++--
 2 files changed, 17 insertions(+), 11 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 5e3e0435b4..e8814c686c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -82,6 +82,7 @@ Copyright @copyright{} 2020 Pierre Langlois@*
 Copyright @copyright{} 2020 pinoaffe@*
 Copyright @copyright{} 2020 André Batista@*
 Copyright @copyright{} 2020 Alexandru-Sergiu Marton@*
+Copyright @copyright{} 2020 John Soo@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -12661,9 +12662,12 @@ ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 LockHeld: /gnu/store/@dots{}-perl-ipc-cmd-0.96.lock
 LockHeld: /gnu/store/@dots{}-python-six-bootstrap-1.11.0.lock
 LockHeld: /gnu/store/@dots{}-libjpeg-turbo-2.0.0.lock
-ChildProcess: 20495: guix offload x86_64-linux 7200 1 28800
-ChildProcess: 27733: guix offload x86_64-linux 7200 1 28800
-ChildProcess: 27793: guix offload x86_64-linux 7200 1 28800
+ChildPID: 20495
+ChildCommand: guix offload x86_64-linux 7200 1 28800
+ChildPID: 27733
+ChildCommand: guix offload x86_64-linux 7200 1 28800
+ChildPID: 27793
+ChildCommand: guix offload x86_64-linux 7200 1 28800
 @end example
 
 In this example we see that @command{guix-daemon} has three clients:
@@ -12672,12 +12676,12 @@ integration tool; their process identifier (PID) is given by the
 @code{ClientPID} field.  The @code{SessionPID} field gives the PID of the
 @command{guix-daemon} sub-process of this particular session.
 
-The @code{LockHeld} fields show which store items are currently locked by this
-session, which corresponds to store items being built or substituted (the
-@code{LockHeld} field is not displayed when @command{guix processes} is not
-running as root).  Last, by looking at the @code{ChildProcess} field, we
-understand that these three builds are being offloaded (@pxref{Daemon Offload
-Setup}).
+The @code{LockHeld} fields show which store items are currently locked
+by this session, which corresponds to store items being built or
+substituted (the @code{LockHeld} field is not displayed when
+@command{guix processes} is not running as root).  Last, by looking at
+the @code{ChildPID} and @code{ChildCommand} fields, we understand that
+these three builds are being offloaded (@pxref{Daemon Offload Setup}).
 
 The output is in Recutils format so we can use the handy @command{recsel}
 command to select sessions of interest (@pxref{Selection Expressions,,,
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index b4ca7b1687..3a7ea0b89c 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2020 John Soo <jsoo1@asu.edu>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -188,8 +189,9 @@ active sessions, and the master 'guix-daemon' process."
               (format port "LockHeld: ~a~%" lock))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
-              (format port "ChildProcess: ~a:~{ ~a~}~%"
-                      (process-id process)
+              (format port "ChildPID: ~a~%"
+                      (process-id process))
+              (format port "ChildCommand: :~{ ~a~}~%"
                       (process-command process)))
             (daemon-session-children session)))
 
-- 
2.29.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-processes-Optionally-normalize-recutils-output.patch --]
[-- Type: text/x-patch, Size: 8472 bytes --]

From becf3a8fee4aea0a49dde47f5728410b97d94fbf Mon Sep 17 00:00:00 2001
From: John Soo <jsoo1@asu.edu>
Date: Wed, 4 Nov 2020 07:51:52 -0800
Subject: [PATCH 2/2] processes: Optionally normalize recutils output.

* guix/scripts/processes.scm: Add "format" and "list-formats" flag.
* doc/guix.texi: Document new flags.
---
 doc/guix.texi              |  38 ++++++++++
 guix/scripts/processes.scm | 144 ++++++++++++++++++++++++++++++++-----
 2 files changed, 163 insertions(+), 19 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e8814c686c..d67ccd8acc 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12695,6 +12695,44 @@ ClientPID: 19419
 ClientCommand: cuirass --cache-directory /var/cache/cuirass @dots{}
 @end example
 
+Additional options are listed below.
+
+@table @code
+@item --format=@var{format}
+@itemx -f @var{format}
+Produce output in the specified @var{format}, one of:
+
+@table @code
+@item recutils
+The default option. It outputs a set of Session recutils records
+that include each ChildProcess as a field.
+
+@item normalized
+Normalize the output records into record sets (@pxref{Record Sets,,,
+recutils, GNU recutils manual}).  Normalizing into record sets allows
+joins across record types.  The example below lists the PID of each
+ChildProcess and the associated PID for Session that spawned the
+ChildProcess where the Session was started using guix build.
+
+@example
+$ guix processes --format=normalized | \
+    recsel \
+    -j Session \
+    -t ChildProcess \
+    -p Session.PID,PID \
+    -e 'Session.ClientCommand ~ "guix build'"
+PID: 4435
+Session_PID: 4278
+
+PID: 4554
+Session_PID: 4278
+
+PID: 4646
+Session_PID: 4278
+@end example
+@end table
+@end table
+
 @node System Configuration
 @chapter System Configuration
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index 3a7ea0b89c..bcc541badb 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -177,6 +177,9 @@ active sessions, and the master 'guix-daemon' process."
     (values (filter-map child-process->session children)
             master)))
 
+(define (lock->recutils lock port)
+  (format port "LockHeld: ~a~%" lock))
+
 (define (daemon-session->recutils session port)
   "Display SESSION information in recutils format on PORT."
   (format port "SessionPID: ~a~%"
@@ -185,8 +188,7 @@ active sessions, and the master 'guix-daemon' process."
           (process-id (daemon-session-client session)))
   (format port "ClientCommand:~{ ~a~}~%"
           (process-command (daemon-session-client session)))
-  (for-each (lambda (lock)
-              (format port "LockHeld: ~a~%" lock))
+  (for-each (lambda (lock) (lock->recutils lock port))
             (daemon-session-locks-held session))
   (for-each (lambda (process)
               (format port "ChildPID: ~a~%"
@@ -195,19 +197,102 @@ active sessions, and the master 'guix-daemon' process."
                       (process-command process)))
             (daemon-session-children session)))
 
+(define (daemon-sessions->recutils port sessions)
+  "Display denormalized SESSIONS information to PORT."
+  (for-each (lambda (session)
+              (daemon-session->recutils session port)
+              (newline port))
+            sessions))
+
+(define session-rec-type
+  "%rec: Session
+%type: PID int
+%type: ClientPID int
+%key: PID
+%mandatory: ClientPID ClientCommand")
+
+(define lock-rec-type
+  "%rec: Lock
+%mandatory: LockHeld
+%type: Session rec Session")
+
+(define child-process-rec-type
+  "%rec: ChildProcess
+%type: PID int
+%type: Session rec Session
+%key: PID
+%mandatory: Command")
+
+(define (session-key->recutils session port)
+  "Display SESSION PID as a recutils field on PORT."
+  (format
+   port "Session: ~a"
+   (process-id (daemon-session-process session))))
+
+(define (session-scalars->normalized-record session port)
+  "Display SESSION scalar fields to PORT in normalized form."
+  (format port "PID: ~a~%"
+          (process-id (daemon-session-process session)))
+  (format port "ClientPID: ~a~%"
+          (process-id (daemon-session-client session)))
+  (format port "ClientCommand:~{ ~a~}~%"
+          (process-command (daemon-session-client session))))
+
+(define (child-process->normalized-record process port)
+  "Display PROCESS record on PORT in normalized form"
+  (format port "PID: ~a" (process-id process))
+  (newline port)
+  (format port "Command:~{ ~a~}" (process-command process)))
+
+(define (daemon-sessions->normalized-record port sessions)
+  "Display SESSIONS recutils on PORT in normalized form"
+  (display session-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (session-scalars->normalized-record session port)
+              (newline port))
+            sessions)
+
+  (display lock-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (lock)
+                          (lock->recutils "testing testing" port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-locks-held session)))
+            sessions)
+
+  (display child-process-rec-type port)
+  (newline port)
+  (newline port)
+  (for-each (lambda (session)
+              (for-each (lambda (process)
+                          (child-process->normalized-record process port)
+                          (newline port)
+                          (session-key->recutils session port)
+                          (newline port)
+                          (newline port))
+                        (daemon-session-children session)))
+            sessions))
+
 \f
 ;;;
 ;;; Options.
 ;;;
 
-(define %options
-  (list (option '(#\h "help") #f #f
-                (lambda args
-                  (show-help)
-                  (exit 0)))
-        (option '(#\V "version") #f #f
-                (lambda args
-                  (show-version-and-exit "guix processes")))))
+(define %available-formats
+  '("recutils" "normalized"))
+
+(define (list-formats)
+  (display (G_ "The available formats are:\n"))
+  (newline)
+  (for-each (lambda (f)
+              (format #t "  - ~a~%" f))
+            %available-formats))
 
 (define (show-help)
   (display (G_ "Usage: guix processes
@@ -218,8 +303,33 @@ List the current Guix sessions and their processes."))
   (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
+  (display (G_ "
+  -f, --format=FORMAT    display results as normalized record sets"))
+  (display (G_ "
+  --list-formats         display available formats"))
+  (newline)
   (show-bug-report-information))
 
+(define %options
+  (list (option '(#\h "help") #f #f
+                (lambda args
+                  (show-help)
+                  (exit 0)))
+        (option '(#\V "version") #f #f
+                (lambda args
+                  (show-version-and-exit "guix processes")))
+        (option '(#\f "format") #t #f
+                (lambda (opt name arg result)
+                  (unless (member arg %available-formats)
+                    (leave (G_ "~a: unsupported output format~%") arg))
+                  (alist-cons 'format (string->symbol arg) result)))
+        (option '("list-formats") #f #f
+                (lambda (opt name arg result)
+                  (list-formats)
+                  (exit 0)))))
+
+(define %default-options '((format . recutils)))
+
 \f
 ;;;
 ;;; Entry point.
@@ -229,17 +339,13 @@ List the current Guix sessions and their processes."))
   (category plumbing)
   (synopsis "list currently running sessions")
   (define options
-    (args-fold* args %options
-                (lambda (opt name arg result)
-                  (leave (G_ "~A: unrecognized option~%") name))
-                cons
-                '()))
+    (parse-command-line args %options (list %default-options)))
 
   (with-paginated-output-port port
-    (for-each (lambda (session)
-                (daemon-session->recutils session port)
-                (newline port))
-              (daemon-sessions))
+    (match (assoc-ref options 'format)
+      ('normalized
+       (daemon-sessions->normalized-record port (daemon-sessions)))
+      (_ (daemon-sessions->recutils port (daemon-sessions))))
 
     ;; Pass 'R' (instead of 'r') so 'less' correctly estimates line length.
     #:less-options "FRX"))
-- 
2.29.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread

* bug#44460: Add copyright lines
  2020-11-13 16:16               ` John Soo
@ 2020-11-29 22:49                 ` Ludovic Courtès
  0 siblings, 0 replies; 14+ messages in thread
From: Ludovic Courtès @ 2020-11-29 22:49 UTC (permalink / raw)
  To: John Soo; +Cc: 44460-done

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

Hi,

John Soo <jsoo1@asu.edu> skribis:

>>From 6082c559d1200e632b3fb45eb0633d28829667a1 Mon Sep 17 00:00:00 2001
> From: John Soo <jsoo1@asu.edu>
> Date: Thu, 12 Nov 2020 21:16:48 -0800
> Subject: [PATCH 1/2] processes: Put ChildProcess and ChildPID on separate
>  lines.
>
> * guix/scripts/processes.scm: Put child process information in
> separate fields.
> * doc/guix.texi: Document change in output of guix processes.

Applied.

>>From becf3a8fee4aea0a49dde47f5728410b97d94fbf Mon Sep 17 00:00:00 2001
> From: John Soo <jsoo1@asu.edu>
> Date: Wed, 4 Nov 2020 07:51:52 -0800
> Subject: [PATCH 2/2] processes: Optionally normalize recutils output.
>
> * guix/scripts/processes.scm: Add "format" and "list-formats" flag.
> * doc/guix.texi: Document new flags.

Applied with the changes below.

I also tweaked the commit logs to list the entities added or modified.

Thanks!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1860 bytes --]

diff --git a/doc/guix.texi b/doc/guix.texi
index b739464853..fcaa2f2b63 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12884,14 +12884,15 @@ Produce output in the specified @var{format}, one of:
 @table @code
 @item recutils
 The default option. It outputs a set of Session recutils records
-that include each ChildProcess as a field.
+that include each @code{ChildProcess} as a field.
 
 @item normalized
 Normalize the output records into record sets (@pxref{Record Sets,,,
 recutils, GNU recutils manual}).  Normalizing into record sets allows
 joins across record types.  The example below lists the PID of each
-ChildProcess and the associated PID for Session that spawned the
-ChildProcess where the Session was started using guix build.
+@code{ChildProcess} and the associated PID for @code{Session} that
+spawned the @code{ChildProcess} where the @code{Session} was started
+using @command{guix build}.
 
 @example
 $ guix processes --format=normalized | \
@@ -12899,7 +12900,7 @@ $ guix processes --format=normalized | \
     -j Session \
     -t ChildProcess \
     -p Session.PID,PID \
-    -e 'Session.ClientCommand ~ "guix build'"
+    -e 'Session.ClientCommand ~ "guix build"'
 PID: 4435
 Session_PID: 4278
 
diff --git a/guix/scripts/processes.scm b/guix/scripts/processes.scm
index bcc541badb..87e687852c 100644
--- a/guix/scripts/processes.scm
+++ b/guix/scripts/processes.scm
@@ -338,8 +338,10 @@ List the current Guix sessions and their processes."))
 (define-command (guix-processes . args)
   (category plumbing)
   (synopsis "list currently running sessions")
+
   (define options
-    (parse-command-line args %options (list %default-options)))
+    (parse-command-line args %options (list %default-options)
+                        #:build-options? #f))
 
   (with-paginated-output-port port
     (match (assoc-ref options 'format)

^ permalink raw reply related	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2020-11-29 22:51 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-11-05  4:31 [bug#44460] [PATCH] processes: Optionally normalize recutils output John Soo
2020-11-05 15:49 ` [bug#44460] processes: Don't normalize Locks John Soo
2020-11-05 16:01 ` [bug#44460] Fixup the output of Session John Soo
2020-11-06 23:34 ` [bug#44460] Add copyright lines John Soo
2020-11-10 22:15   ` Ludovic Courtès
2020-11-11 17:51     ` John Soo
2020-11-11 17:59       ` John Soo
2020-11-11 18:47         ` John Soo
2020-11-12 10:58       ` Ludovic Courtès
2020-11-12 15:37         ` John Soo
2020-11-12 20:29           ` Ludovic Courtès
2020-11-13  5:33             ` John Soo
2020-11-13 16:16               ` John Soo
2020-11-29 22:49                 ` bug#44460: " Ludovic Courtès

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.