Announcing shepherd-run

Announcing shepherd-run

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 899 bytes --]

Announcing the initial release of shepherd-run!

Do you have experience with systemd and systemd-run? Do you wish you
were able to quickly and easily add a simple service to your (user or
system) shepherd instance?

Using the finest technologies from the 70's, written in gawk, I present
shepherd-run!

Want to build a derivation but don't care when it finishes?
shepherd-run -e 'guix build /gnu/store/...-drv'

Need to clear up some room in your store but don't want to dedicate a
terminal to waiting for it to finish?
shepherd-run -e "guix gc"

Want to spam your shepherd logs?
'shepherd run --exec yes'

Get ready, it's coming soon to a guix repository near you!

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: 0001-gnu-Add-shepherd-run.patch --]
[-- Type: text/plain, Size: 3508 bytes --]

From 0987a56039f9626664f89095b292421081daddae Mon Sep 17 00:00:00 2001
Message-ID: <0987a56039f9626664f89095b292421081daddae.1730639824.git.efraim@flashner.co.il>
From: Efraim Flashner <efraim@flashner.co.il>
Date: Sun, 3 Nov 2024 15:16:11 +0200
Subject: [PATCH] gnu: Add shepherd-run.

* gnu/packages/admin.scm (shepherd-run): New variable.

Change-Id: I4b08c84f83bcc588db7a11619321f4c8dee48257
---
 gnu/packages/admin.scm | 42 +++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index 88d68a3afcc..575b00736fe 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -9,7 +9,7 @@
 ;;; Copyright © 2016, 2017, 2020 Leo Famulari <leo@famulari.name>
 ;;; Copyright © 2016 Pjotr Prins <pjotr.guix@thebird.nl>
 ;;; Copyright © 2016, 2017, 2023 Ricardo Wurmus <rekado@elephly.net>
-;;; Copyright © 2016-2023 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2016-2024 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2016 Peter Feigl <peter.feigl@nexoid.at>
 ;;; Copyright © 2016 John J. Foerch <jjfoerch@earthlink.net>
 ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is>
@@ -423,6 +423,46 @@ (define-public guile2.2-shepherd
     (native-inputs (list pkg-config guile-2.2))
     (inputs (list guile-2.2 guile2.2-fibers))))
 
+(define-public shepherd-run
+  (package
+    (name "shepherd-run")
+    (version "0.1.0")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://git.sr.ht/~efraim/shepherd-run")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "033l8ignsrr6p2wgwcyqlswpbf58kyl8cf7zwkz028gqfq4arkr8"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list
+       #:make-flags #~(list (string-append "PREFIX=" #$output))
+       #:phases
+       #~(modify-phases %standard-phases
+           (delete 'configure)          ; No configure script.
+           ;; First 'check checks the shell script which loads the gawk code.
+           ;; This 'check checks the installed gawk script.
+           (add-after 'patch-shebangs 'check-again
+             (lambda args
+               (apply (assoc-ref %standard-phases 'check)
+                      (append args
+                              (list #:make-flags
+                                    (list (string-append "BINARY=" %output
+                                                         "/bin/shepherd-run"))))))))))
+    (native-inputs (list diffutils help2man))
+    (inputs (list gawk))
+    (synopsis "Create GNU Shepherd services from the command line")
+    (description
+     "Shepherd-run is a script which assists in creating one-off shepherd
+services from the command line.  It is meant to partially fill the void left
+by @command{systemd-run}, since GNU Guix uses GNU Shepherd as its system service
+manager.")
+    (home-page "https://git.sr.ht/~efraim/shepherd-run")
+    (license license:gpl3+)))
+
 (define-public swineherd
   (package
     (name "swineherd")

base-commit: 8964dfdb84f7d21dbc89c217ca4f4546a15990af
-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Announcing shepherd-run

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 988 bytes --]

Hi,

On Sun, 3 Nov 2024 15:19:04 +0200
Efraim Flashner <efraim@flashner.co.il> wrote:

> Announcing the initial release of shepherd-run!
> 
> Do you have experience with systemd and systemd-run? Do you wish you
> were able to quickly and easily add a simple service to your (user or
> system) shepherd instance?
> 
> Using the finest technologies from the 70's, written in gawk, I
> present shepherd-run!
Is there any plans to integrate that with Guix and Shepherd later on?

I'm thinking of uses cases like netctl which is an utility from Arch
Linux that is present in Parabola. Users define network profiles and
netctl somehow interact with systemd to setup the network.

Here for instance Guix system services or Guix home services are not
dynamic, and especially for the network it could make sense to have
different profiles.

As I understand netctl generates systemd unit files, so there is
also some similarities between netctl and shepherd-run.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Announcing shepherd-run

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1723 bytes --]

On Sun, Nov 03, 2024 at 04:57:09PM +0100, Denis 'GNUtoo' Carikli wrote:
> Hi,
> 
> On Sun, 3 Nov 2024 15:19:04 +0200
> Efraim Flashner <efraim@flashner.co.il> wrote:
> 
> > Announcing the initial release of shepherd-run!
> > 
> > Do you have experience with systemd and systemd-run? Do you wish you
> > were able to quickly and easily add a simple service to your (user or
> > system) shepherd instance?
> > 
> > Using the finest technologies from the 70's, written in gawk, I
> > present shepherd-run!
> Is there any plans to integrate that with Guix and Shepherd later on?

If it proves to be useful I could see re-writing it in guile so it can
be part of shepherd directly.

> I'm thinking of uses cases like netctl which is an utility from Arch
> Linux that is present in Parabola. Users define network profiles and
> netctl somehow interact with systemd to setup the network.
> 
> Here for instance Guix system services or Guix home services are not
> dynamic, and especially for the network it could make sense to have
> different profiles.
> 
> As I understand netctl generates systemd unit files, so there is
> also some similarities between netctl and shepherd-run.
> 
> Denis.

Looking at some of the manpages for systemd I saw they have options for
on-joining or leaving certain networks and other triggers. I don't
believe shepherd currently has the ability to detect that directly
currently, but I suppose it would be possible to imitate that
functionality.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Announcing shepherd-run

[Ludovic Courtès]

Hello,

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Sun, Nov 03, 2024 at 04:57:09PM +0100, Denis 'GNUtoo' Carikli wrote:
>> Hi,
>> 
>> On Sun, 3 Nov 2024 15:19:04 +0200
>> Efraim Flashner <efraim@flashner.co.il> wrote:
>> 
>> > Announcing the initial release of shepherd-run!
>> > 
>> > Do you have experience with systemd and systemd-run? Do you wish you
>> > were able to quickly and easily add a simple service to your (user or
>> > system) shepherd instance?
>> > 
>> > Using the finest technologies from the 70's, written in gawk, I
>> > present shepherd-run!

An interesting blend of technologies from the 70’s.  :-)

>> Is there any plans to integrate that with Guix and Shepherd later on?
>
> If it proves to be useful I could see re-writing it in guile so it can
> be part of shepherd directly.

Shepherd 1.0-to-be (current ‘devel’ branch) gained a generic ‘timer’
service that does the job the ‘at’ command would do back in the day
(before the advent of CD-ROM and the information superhighway):

  herd schedule timer at 07:00 -- mpg123 alarm.mp3

We’re talking about something quite similar, just not delayed, right?

Ludo’.

Re: Announcing shepherd-run

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2144 bytes --]

On Mon, Nov 04, 2024 at 11:34:50PM +0100, Ludovic Courtès wrote:
> Hello,
> 
> Efraim Flashner <efraim@flashner.co.il> skribis:
> 
> > On Sun, Nov 03, 2024 at 04:57:09PM +0100, Denis 'GNUtoo' Carikli wrote:
> >> Hi,
> >> 
> >> On Sun, 3 Nov 2024 15:19:04 +0200
> >> Efraim Flashner <efraim@flashner.co.il> wrote:
> >> 
> >> > Announcing the initial release of shepherd-run!
> >> > 
> >> > Do you have experience with systemd and systemd-run? Do you wish you
> >> > were able to quickly and easily add a simple service to your (user or
> >> > system) shepherd instance?
> >> > 
> >> > Using the finest technologies from the 70's, written in gawk, I
> >> > present shepherd-run!
> 
> An interesting blend of technologies from the 70’s.  :-)
> 
> >> Is there any plans to integrate that with Guix and Shepherd later on?
> >
> > If it proves to be useful I could see re-writing it in guile so it can
> > be part of shepherd directly.
> 
> Shepherd 1.0-to-be (current ‘devel’ branch) gained a generic ‘timer’
> service that does the job the ‘at’ command would do back in the day
> (before the advent of CD-ROM and the information superhighway):
> 
>   herd schedule timer at 07:00 -- mpg123 alarm.mp3
> 
> We’re talking about something quite similar, just not delayed, right?

systemd-run has a --timer flag, and I could see adding a flag for
shepherd-run after shepherd 1.0.

To me the big thing systemd-run brought was being able to launch
programs from the shell and have it show up in `systemctl --user
services`, and I'd like to be able to get commands I run from the CLI to
show up in `herd status` the same way.

systemd has some more magic I wanted to steal, like being able to launch
GUI applications from systemd-run.

shepherd-run basically parses the flags and commands and then writes out
a service to a temporary file and loads it with 'herd load root ...'

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Announcing shepherd-run

[Ludovic Courtès]

Efraim Flashner <efraim@flashner.co.il> skribis:

> On Mon, Nov 04, 2024 at 11:34:50PM +0100, Ludovic Courtès wrote:

[...]

>> Shepherd 1.0-to-be (current ‘devel’ branch) gained a generic ‘timer’
>> service that does the job the ‘at’ command would do back in the day
>> (before the advent of CD-ROM and the information superhighway):
>> 
>>   herd schedule timer at 07:00 -- mpg123 alarm.mp3
>> 
>> We’re talking about something quite similar, just not delayed, right?
>
> systemd-run has a --timer flag, and I could see adding a flag for
> shepherd-run after shepherd 1.0.
>
> To me the big thing systemd-run brought was being able to launch
> programs from the shell and have it show up in `systemctl --user
> services`, and I'd like to be able to get commands I run from the CLI to
> show up in `herd status` the same way.

Easy.  :-)  That’s really what ‘herd schedule timer’ does, except it’s
delayed.

> systemd has some more magic I wanted to steal, like being able to launch
> GUI applications from systemd-run.

Trickier I guess, because you need to pass DISPLAY etc.

Food for thought!

Ludo’.

Re: Announcing shepherd-run

[Runciter via Development of GNU Guix and the GNU System distribution.]

Ludovic Courtès <ludo@gnu.org> writes:

Hello Ludo,

>   herd schedule timer at 07:00 -- mpg123 alarm.mp3

Sorry for hijacking the thread a little, and I haven't tried the
Shepherd release candidate yet.

Not that this is any of shepherd or herd's problem, but if you have
included or plan to include this example in documentation, you might want
to reconsider and include another simple meaningful example instead.

I possibly noticed this only because I use a minimal WM: mpg123 won't
start pulseaudio if pulseaudio is not already running. If there is a
workaround other than starting pulseaudio prior, I don't know it. In
case it doesn't ring, the evidence that herd did something at all will
be buried in the logs.

I actually think this is cool though, and the rest is mostly random
thoughts now that I think about this problem: a herd-based minimalist
alarm clock for production needs 2 features and must not include a third
feature:

* Repeat, but this is trivial to do with a script.

* A heuristic to output sound at all costs, while trying hard to output
  through speakers and not headphones. This is harder, maybe not
  absolutely required; I suppose phones get it right but I'm not
  even sure.

* A keystroke to stop the alarm from the X lock screen; surely the
  significant other will close their mind to the truth that this feature
  increases the attack surface and must not be included.

Re: Announcing shepherd-run

[Ludovic Courtès]

Runciter via "Development of GNU Guix and the GNU System distribution."
<guix-devel@gnu.org> skribis:

> I actually think this is cool though, and the rest is mostly random
> thoughts now that I think about this problem: a herd-based minimalist
> alarm clock for production needs 2 features and must not include a third
> feature:
>
> * Repeat, but this is trivial to do with a script.
>
> * A heuristic to output sound at all costs, while trying hard to output
>   through speakers and not headphones. This is harder, maybe not
>   absolutely required; I suppose phones get it right but I'm not
>   even sure.
>
> * A keystroke to stop the alarm from the X lock screen; surely the
>   significant other will close their mind to the truth that this feature
>   increases the attack surface and must not be included.

All good points!  I’d like to have this, but I think it’s beyond the
scope of the ‘timer’ feature.  :-)

Ludo’.

Re: Announcing shepherd-run

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 570 bytes --]

Thanks to a user bug report I discovered that the Shepherd devs (cough
Ludo cough) were serious about removing the GOOPS interface and
shepherd-run was outputting a deprecated service declaration which
wasn't compatible with the 1.0.0rc1 tag of shepherd.

0.2.0 is now out!
100% fewer GOOPS!
'shepherd-run --version' now returns the correct string!

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Announcing shepherd-run

[Ludovic Courtès]

Hi!

Efraim Flashner <efraim@flashner.co.il> skribis:

> 0.2.0 is now out!
> 100% fewer GOOPS!
> 'shepherd-run --version' now returns the correct string!

Wo0t!

Inspired by shepherd-run, I thought “hey let’s add this to Shepherd
proper for 1.0, easy!”.  Turns out that finding a way to pass the
various options (‘-d’, ‘-E’, etc.) to the action that spawns a service
was still an open issue.  Solution: add these options to ‘herd’, and
have it pass them as keyword arguments to the action that’s invoked.

That’s allowed me to unify ‘herd schedule timer’ and the new ‘herd spawn
transient’ (the name of the Awk-less GOOPS-less C-less ‘systemd-run’).
Here’s the doc:

--8<---------------cut here---------------start------------->8---
The “transient service maker” lets you run commands in the background,
and it does so by wrapping those commands in transient services (*note
Defining Services::).  It is similar to the ‘systemd-run’
(https://www.freedesktop.org/software/systemd/man/latest/systemd-run.html)
command, which you might have encountered before.

   A simple configuration file that uses this service looks like this:

     (use-modules (shepherd service transient))

     (register-services (list (transient-service)))

   This creates a service called ‘transient’ that has a ‘spawn’ action,
which you can use like this:

     # Run 'make' from the current directory.
     herd spawn transient -d "$PWD" -- make -j4

     # Run 'rsync' from the home directory, inheriting
     # the 'SSH_AUTH_SOCK' environment variable.
     herd spawn transient \
       --log-file=backup.log \
       -E SSH_AUTH_SOCK=$SSH_AUTH_SOCK -- \
       rsync -e ssh -vur . backup.example.org:

   Each of these ‘herd spawn transient’ commands creates a new transient
service.  Like any other service, they can be inspected and stopped.
Running ‘herd stop transient’ stops all the currently running
transients.

   The command runs from the directory specified by
‘default-service-directory’ or from that specified by the
‘--working-directory’ option of ‘herd’; it has with the environment
variables in ‘default-environment-variables’, augmented with ‘HOME’ when
running as a different user, with the addition of variables passed with
‘--environment-variable’.  *Note Invoking herd::, for more info on
influential command-line options.

 -- Procedure: transient-service [PROVISION] [#:requirement '()]
     Return a service with the given PROVISION and REQUIREMENT.  The
     service has a ‘spawn’ action that lets users run commands in the
     background.
--8<---------------cut here---------------end--------------->8---

It’s too big a change in between two release candidates, but it seemed
better to get that right before 1.0.

Lemme know what you think!

Ludo’.