The 🐑 Shepherd gets a service collection

The 🐑 Shepherd gets a service collection

[Ludovic Courtès]

Hello Guix!

I pushed some changes yesterday that confirm that the Shepherd paves the
way for init system innovation, synergistic cross-domain fertilization,
and delimited continuations:

  https://git.savannah.gnu.org/cgit/shepherd.git/log/?id=31d21fa083872d500c016b6b3b2587d25510702d

  31d21fa * Add REPL service.
  cd6f3fb * comm: Add 'open-server-socket'.
  c64804f * Add resource monitoring service.

These new services are built into shepherd, allowing users to control it
and to fiddle with it.  The REPL is functional but of course a bit
bumpy: you’d better know what you’re doing.

I imagine we could develop more convenient services like this, such as a
basic command scheduler similar to the ‘at’ command, and a syslogd
implementation.  The latter could be nice for a couple of reasons:
logging would happen from the start and till the end (an improvement
over the external syslogd process), and it could let us provide a nicer
user interface to view logs (taking inspiration from that of
‘journalctl’).

Thoughts?  Ideas?

Ludo’.

PS: It just occurred to me that we might as well rename the new
    (shepherd service …) hierarchy to (shepherd sheep …) or even
    (shepherd 🐑 …).

Re: The 🐑 Shepherd gets a service collection

[indieterminacy]

Thanks!
> 
> PS: It just occurred to me that we might as well rename the new
>     (shepherd service …) hierarchy to (shepherd sheep …) or even
>     (shepherd 🐑 …).

Its a shame that Directed Acyclic Graphs are part of Guix verbiage,
as otherwise Id be suggesting the name 'dags'.

(Dags are the yellowed clumps of wool that cluster around their 
netherregions.
There is a antipodean phrase, 'rattle yer dags!', to mean 'hurry up!', 
given
the clumps bouncing around when sheep move quickly.

-- 
Jonathan McHugh
indieterminacy@libre.brussels

Re: The 🐑 Shepherd gets a service collection

[Adam Faiz]

> I imagine we could develop more convenient services like this, such as 
> basic command scheduler similar to the ‘at’ command, and a syslogd
> implementation.  The latter could be nice for a couple of reasons:
> logging would happen from the start and till the end (an improvement
> over the external syslogd process), and it could let us provide a nicer
> user interface to view logs (taking inspiration from that of
> ‘journalctl’).
> 
> Thoughts?  Ideas?I don't think a command scheduler should be added to the Shepherd, isn't that what mcron does?
If mcron has any deficiencies for being used as an `at` command, then that can be improved.

Regarding syslogd, I think a better approach is to tell the services to send their output to stdout and stderror,
so that logs don't depend on a separate logging service in the first place.
Per-service logging is already implemented in the Shepherd, but could be streamlined to have a default logs directory:
https://skarnet.org/software/s6/s6-log.html#loggingchain

I'm already happy with zless/any GUI log viewer, but it would be great to have a nicer user interface to view logs nonetheless.

Re: The 🐑 Shepherd gets a service collection

[Maxim Cournoyer]

Hi Ludo!

Ludovic Courtès <ludo@gnu.org> writes:

> Hello Guix!
>
> I pushed some changes yesterday that confirm that the Shepherd paves the
> way for init system innovation, synergistic cross-domain fertilization,
> and delimited continuations:
>
>   https://git.savannah.gnu.org/cgit/shepherd.git/log/?id=31d21fa083872d500c016b6b3b2587d25510702d
>
>   31d21fa * Add REPL service.
>   cd6f3fb * comm: Add 'open-server-socket'.
>   c64804f * Add resource monitoring service.
>
> These new services are built into shepherd, allowing users to control it
> and to fiddle with it.  The REPL is functional but of course a bit
> bumpy: you’d better know what you’re doing.
>
> I imagine we could develop more convenient services like this, such as a
> basic command scheduler similar to the ‘at’ command, and a syslogd
> implementation.  The latter could be nice for a couple of reasons:
> logging would happen from the start and till the end (an improvement
> over the external syslogd process), and it could let us provide a nicer
> user interface to view logs (taking inspiration from that of
> ‘journalctl’).
>
> Thoughts?  Ideas?

While I also find the journalctl interface to be convenient, the
underlying database logs is costly in terms of storage and complexity (I
remember comparing it to a simple, non-compressed text file, and to my
surprise the later used less space even the systemd logs are supposed to
be compressed, if I remember correctly).  I think it had to do with all
the keys having to be stored for each message, even when they don't
contain anything useful.

It'd be nice if Shepherd didn't end up with that same caveat, should
database logs be implemented.

Thanks for improving Shepherd!

-- 
Maxim

Re: The 🐑 Shepherd gets a service collection

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1743 bytes --]

On Mon, Mar 13, 2023 at 05:11:34PM +0100, Ludovic Courtès wrote:
> Hello Guix!
> 
> I pushed some changes yesterday that confirm that the Shepherd paves the
> way for init system innovation, synergistic cross-domain fertilization,
> and delimited continuations:
> 
>   https://git.savannah.gnu.org/cgit/shepherd.git/log/?id=31d21fa083872d500c016b6b3b2587d25510702d
> 
>   31d21fa * Add REPL service.
>   cd6f3fb * comm: Add 'open-server-socket'.
>   c64804f * Add resource monitoring service.
> 
> These new services are built into shepherd, allowing users to control it
> and to fiddle with it.  The REPL is functional but of course a bit
> bumpy: you’d better know what you’re doing.
> 
> I imagine we could develop more convenient services like this, such as a
> basic command scheduler similar to the ‘at’ command, and a syslogd
> implementation.  The latter could be nice for a couple of reasons:
> logging would happen from the start and till the end (an improvement
> over the external syslogd process), and it could let us provide a nicer
> user interface to view logs (taking inspiration from that of
> ‘journalctl’).
> 
> Thoughts?  Ideas?

I always imagined an 'at' command being part of mcron. Does this mean we
should look forward to shepherdd-mcron?

> PS: It just occurred to me that we might as well rename the new
>     (shepherd service …) hierarchy to (shepherd sheep …) or even
>     (shepherd 🐑 …).

How does shepherd handle emojis for system symbols?

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: The 🐑 Shepherd gets a service collection

[Ludovic Courtès]

Hi,

Adam Faiz <adam.faiz@disroot.org> skribis:

>> I imagine we could develop more convenient services like this, such
>> as basic command scheduler similar to the ‘at’ command, and a
>> syslogd
>> implementation.  The latter could be nice for a couple of reasons:
>> logging would happen from the start and till the end (an improvement
>> over the external syslogd process), and it could let us provide a nicer
>> user interface to view logs (taking inspiration from that of
>> ‘journalctl’).
>> Thoughts?  Ideas?I don't think a command scheduler should be added
>> to the Shepherd, isn't that what mcron does?
> If mcron has any deficiencies for being used as an `at` command, then that can be improved.

The main limitation of mcron for such thing is that it’s entirely
static: it reads a list of job specs upfront and then goes on to
schedule them.  There’s no communication protocol to talk to it and
add/remove jobs on the fly, which is what ‘at’ would need.

> Regarding syslogd, I think a better approach is to tell the services to send their output to stdout and stderror,
> so that logs don't depend on a separate logging service in the first place.

Yes, but:

  1. Some daemons include syslog support even today, sometimes optional,
     sometimes mandatory.

  2. Syslog is a bit more structured than just stdout/stderr output:
     there are facilities and levels, for instance—see syslog(3);
     syslogd provides interesting filtering capabilities.

> Per-service logging is already implemented in the Shepherd, but could be streamlined to have a default logs directory:
> https://skarnet.org/software/s6/s6-log.html#loggingchain

Interesting read, thanks!

Regarding the default logs directory, there’s /var/log already, or did
you mean something else?

Thanks,
Ludo’.

Re: The 🐑 Shepherd gets a service collection

[Ludovic Courtès]

Hi,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> I imagine we could develop more convenient services like this, such as a
>> basic command scheduler similar to the ‘at’ command, and a syslogd
>> implementation.  The latter could be nice for a couple of reasons:
>> logging would happen from the start and till the end (an improvement
>> over the external syslogd process), and it could let us provide a nicer
>> user interface to view logs (taking inspiration from that of
>> ‘journalctl’).
>>
>> Thoughts?  Ideas?
>
> While I also find the journalctl interface to be convenient, the
> underlying database logs is costly in terms of storage and complexity

Agreed (that’s why I mentioned the user interface of ‘journalctl’, not
the storage mechanism of ‘journald’).

Ludo’.

Re: The 🐑 Shepherd gets a service collection

[Adam Faiz]

On 3/16/23 22:14, Ludovic Courtès wrote:
> The main limitation of mcron for such thing is that it’s entirely
> static: it reads a list of job specs upfront and then goes on to
> schedule them.  There’s no communication protocol to talk to it and
> add/remove jobs on the fly, which is what ‘at’ would need.
Would it be easier to add dynamic job spec support to mcron than adding a new command scheduler?

>> Regarding syslogd, I think a better approach is to tell the services to send their output to stdout and stderror,
>> so that logs don't depend on a separate logging service in the first place.
> 
> Yes, but:
> 
>    1. Some daemons include syslog support even today, sometimes optional,
>       sometimes mandatory.
> 
>    2. Syslog is a bit more structured than just stdout/stderr output:
>       there are facilities and levels, for instance—see syslog(3);
>       syslogd provides interesting filtering capabilities.
>
Thanks, it looks like syslog is still important for structured logs.

Are there issues of logs sent to syslog being lost even when the syslogd service is specified as a requirement of services that use it?
If not, I think it's not necessary to add a syslogd implementation to the shepherd.
  
>> Per-service logging is already implemented in the Shepherd, but could be streamlined to have a default logs directory:
>> https://skarnet.org/software/s6/s6-log.html#loggingchain
> 
> Interesting read, thanks!
> 
> Regarding the default logs directory, there’s /var/log already, or did
> you mean something else?
I do mean /var/log, I felt like #:log-file in make-forkexec-constructor could be improved.
Rather than always having to specify the absolute log file path, #:log-file could just be set as #t and would then default to /var/log/$canonical-name of the service.
  
> Thanks,
> Ludo’.

Re: The 🐑 Shepherd gets a service collection

[Csepp]

Adam Faiz <adam.faiz@disroot.org> writes:

> On 3/16/23 22:14, Ludovic Courtès wrote:
>> The main limitation of mcron for such thing is that it’s entirely
>> static: it reads a list of job specs upfront and then goes on to
>> schedule them.  There’s no communication protocol to talk to it and
>> add/remove jobs on the fly, which is what ‘at’ would need.
> Would it be easier to add dynamic job spec support to mcron than adding a new command scheduler?

Adding timers to Shepherd means Shepherd services can depend on timers
and vice-versa.

But maybe Shepherd could still reuse mcron code internally.

>>> Regarding syslogd, I think a better approach is to tell the services to send their output to stdout and stderror,
>>> so that logs don't depend on a separate logging service in the first place.
>> Yes, but:
>>    1. Some daemons include syslog support even today, sometimes
>> optional,
>>       sometimes mandatory.
>>    2. Syslog is a bit more structured than just stdout/stderr
>> output:
>>       there are facilities and levels, for instance—see syslog(3);
>>       syslogd provides interesting filtering capabilities.
>>
> Thanks, it looks like syslog is still important for structured logs.
>
> Are there issues of logs sent to syslog being lost even when the syslogd service is specified as a requirement of services that use it?
> If not, I think it's not necessary to add a syslogd implementation to the shepherd.
>  >> Per-service logging is already implemented in the Shepherd, but
>    could be streamlined to have a default logs directory:
>>> https://skarnet.org/software/s6/s6-log.html#loggingchain
>> Interesting read, thanks!
>> Regarding the default logs directory, there’s /var/log already, or
>> did
>> you mean something else?
> I do mean /var/log, I felt like #:log-file in make-forkexec-constructor could be improved.
> Rather than always having to specify the absolute log file path,
> #:log-file could just be set as #t and would then default to
> /var/log/$canonical-name of the service.

That is problematic on flash storage and read-only storage.
Syslog has the advantage of working very nicely in memory only.
Also the structure is very nice, you get to see every event on the
system in a chronological order.  You don't get that with log files.
Honestly /var/log should be redirectable to syslog.

Re: The 🐑 Shepherd gets a service collection

[Maxim Cournoyer]

Hi Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> I imagine we could develop more convenient services like this, such as a
>>> basic command scheduler similar to the ‘at’ command, and a syslogd
>>> implementation.  The latter could be nice for a couple of reasons:
>>> logging would happen from the start and till the end (an improvement
>>> over the external syslogd process), and it could let us provide a nicer
>>> user interface to view logs (taking inspiration from that of
>>> ‘journalctl’).
>>>
>>> Thoughts?  Ideas?
>>
>> While I also find the journalctl interface to be convenient, the
>> underlying database logs is costly in terms of storage and complexity
>
> Agreed (that’s why I mentioned the user interface of ‘journalctl’, not
> the storage mechanism of ‘journald’).

OK. In my mind both are probably hard to separate though (database logs
are more malleable, which is what journald exploits).

-- 
Thanks,
Maxim