[bug#30464] [PATCH 0/2] Add console-agetty-service.

[bug#30464] [PATCH 0/2] Add console-agetty-service.

[Danny Milosavljevic]

Danny Milosavljevic (2):
  services: Add console-agetty-service.
  services: agetty-shepherd-service: Default to providing  'term-console
    instead of 'term-auto .

 gnu/services/base.scm | 38 ++++++++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Danny Milosavljevic]

* gnu/services/base.scm (<agetty-configuration>): Make tty default to #f.
(console-agetty-shepherd-service): New variable.
(console-agetty-service-type): New variable.
(console-agetty-service): New variable.
(%base-services): Replace agetty-service instance by console-agetty-service
instance.
---
 gnu/services/base.scm | 35 ++++++++++++++++++++++++++++++-----
 1 file changed, 30 insertions(+), 5 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 1ad7c2320..29ae5fca3 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -817,7 +817,8 @@ the message of the day, among other things."
   agetty-configuration?
   (agetty           agetty-configuration-agetty   ;<package>
                     (default util-linux))
-  (tty              agetty-configuration-tty)     ;string | #f
+  (tty              agetty-configuration-tty      ;string | #f
+                    (default #f))
   (term             agetty-term                   ;string | #f
                     (default #f))
   (baud-rate        agetty-baud-rate              ;string | #f
@@ -1072,6 +1073,31 @@ program.")))
 the tty to run, among other things."
   (service agetty-service-type config))
 
+(define console-agetty-shepherd-service
+  (lambda (config)
+    (let ((base-service
+           (car
+            (agetty-shepherd-service
+             (agetty-configuration (inherit config)
+              (keep-baud? #t))))))
+      (list
+       (shepherd-service (inherit base-service)
+         ;; Make sure that Linux doesn't log to the same console as well.
+         (requirement (cons 'syslogd (shepherd-service-requirement base-service))))))))
+
+(define console-agetty-service-type
+  (service-type (name 'console-agetty)
+                (extensions (list (service-extension shepherd-root-service-type
+                                                     console-agetty-shepherd-service)))
+                (description
+                 "Provide console login using the @command{agetty}
+program on the Linux console.")))
+
+(define* (console-agetty-service config)
+  "Return a service to run agetty according to @var{config}, which specifies
+the tty to run, defaulting to the Linux console."
+  (service console-agetty-service-type config))
+
 (define-record-type* <mingetty-configuration>
   mingetty-configuration make-mingetty-configuration
   mingetty-configuration?
@@ -2053,10 +2079,9 @@ This service is not part of @var{%base-services}."
                         (cons tty %default-console-font))
                       '("tty1" "tty2" "tty3" "tty4" "tty5" "tty6")))
 
-        (agetty-service (agetty-configuration
-                         (extra-options '("-L")) ; no carrier detect
-                         (term "vt100")
-                         (tty #f))) ; automatic
+        (console-agetty-service (agetty-configuration
+                                 (extra-options '("-L")) ; no carrier detect
+                                 (term "vt100")))
 
         (mingetty-service (mingetty-configuration
                            (tty "tty1")))

[bug#30464] [PATCH 2/2] services: agetty-shepherd-service: Default to providing 'term-console instead of 'term-auto .

[Danny Milosavljevic]

* gnu/services/base.scm (agetty-shepherd-service): Default to providing
'term-console instead of 'term-auto .
---
 gnu/services/base.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 29ae5fca3..4ab83d1d2 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -937,7 +937,8 @@ to use as the tty.  This is primarily useful for headless systems."
        (shepherd-service
          (modules '((ice-9 match) (gnu build linux-boot)))
          (documentation "Run agetty on a tty.")
-         (provision (list (symbol-append 'term- (string->symbol (or tty "auto")))))
+         (provision (list (symbol-append 'term- (string->symbol
+                                                 (or tty "console")))))
 
          ;; Since the login prompt shows the host name, wait for the 'host-name'
          ;; service to be done.  Also wait for udev essentially so that the tty

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Ludovic Courtès]

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> * gnu/services/base.scm (<agetty-configuration>): Make tty default to #f.
> (console-agetty-shepherd-service): New variable.
> (console-agetty-service-type): New variable.
> (console-agetty-service): New variable.
> (%base-services): Replace agetty-service instance by console-agetty-service
> instance.

Could you explain the rationale?

The only difference is that this shepherd service depends on ‘syslogd’
whereas the current agetty service doesn’t, right?

> +(define* (console-agetty-service config)
> +  "Return a service to run agetty according to @var{config}, which specifies
> +the tty to run, defaulting to the Linux console."
> +  (service console-agetty-service-type config))

In general we don’t provide such procedures any more and instead expose
service types and configs directly.

Thanks,
Ludo’.

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Danny Milosavljevic]

Hi Ludo,

On Thu, 15 Feb 2018 15:40:14 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Could you explain the rationale?
> 
> The only difference is that this shepherd service depends on ‘syslogd’
> whereas the current agetty service doesn’t, right?

Yes, it's my inner OCD.

Right now, the agetty is started while the Linux kernel is still booting and
printing stuff to the console.  So you have two parties printing stuff
to the same console simultaneously.  That looks ... not nice.

What this would do is the getty would only be started (and printing stuff)
after syslogd.  The first thing syslogd does is it disables the Linux direct
console printing.

It doesn't make sense to have agetty itself depend on syslogd because
agetty can be (and usually is) used for modems etc which have nothing
to do with the local console.

> > +(define* (console-agetty-service config)
> > +  "Return a service to run agetty according to @var{config}, which specifies
> > +the tty to run, defaulting to the Linux console."
> > +  (service console-agetty-service-type config))  
> 
> In general we don’t provide such procedures any more and instead expose
> service types and configs directly.

Okay, I can remove that one and inline it at the (only) call site.

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Ludovic Courtès]

Heya,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Thu, 15 Feb 2018 15:40:14 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Could you explain the rationale?
>> 
>> The only difference is that this shepherd service depends on ‘syslogd’
>> whereas the current agetty service doesn’t, right?
>
> Yes, it's my inner OCD.
>
> Right now, the agetty is started while the Linux kernel is still booting and
> printing stuff to the console.  So you have two parties printing stuff
> to the same console simultaneously.  That looks ... not nice.

Right.

> What this would do is the getty would only be started (and printing stuff)
> after syslogd.  The first thing syslogd does is it disables the Linux direct
> console printing.

OK, got it.

(Which has me thinking that longer term it’d be nice to have the
Shepherd take care of syslogd-ish activity.)

> It doesn't make sense to have agetty itself depend on syslogd because
> agetty can be (and usually is) used for modems etc which have nothing
> to do with the local console.

Alright.

How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
default to the empty list, and could be set to '(syslogd) in this case.

Does that sound too obscure to you, or would it be OK?

Thanks,
Ludo’.

[bug#30464] shepherd logging; console-agetty-service

[Danny Milosavljevic]

Hi Ludo,

On Thu, 15 Feb 2018 16:47:54 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> (Which has me thinking that longer term it’d be nice to have the
> Shepherd take care of syslogd-ish activity.)

If you mean that we should make sure to log shepherd's own messages, I agree.

Integrating syslogd into shepherd, not sure.  I think external syslogd is fine.
If not, there are a lot of other logging programs to choose from.
I like modularity.

shepherd could write its messages to the kernel log ringbuffer in /dev/kmsg [3].
That sounds dirty, but it would synchronize messages oh-so-nicely and would
not immediately require syslogd.  It would also make sure that syslogd
eventually picks shepherd's messages up (right now they are somewhere on the
first terminal - if you are lucky and they didn't scroll off).

I'm not sure whether then they would be printed to /dev/console as well then -
probably.

We'd need a guile soft-port, but it's not like I haven't done that before.

User-shepherd shouldn't do it though (and can't because it doesn't have
permission to write to /dev/kmsg).

Please stop me and tell me why it's a bad idea :)

Also a way of capturing stderr and stdout (and maybe even /dev/log) of services
would be nice.

If you thought that the above was bad, you ain't seen nothing yet :->

We could also instead open /dev/klog and dup2 its fd to 1 and 2.
That way, shepherd messages and all stray messages by any process shepherd
started will end up in the kernel log.  (problem: there are some reserved
patterns that have special meaning - and we don't control what the services
do as well as we do what just shepherd does)

Also, I know one is supposed to write UNIX services as daemons, but that's
not really composable and kinda complicated to debug for no good reason.
I'd prefer if shepherd also keeps a way to run regular programs as services,
making sure that they are session leader, their output is logged, they are
kept alive and monitored.

daemontools[1][2] have done all this stuff already and I like it much more
than traditional service managers.  It's much more modular, handles logging
on its own, handles error cases well, uses the file system well etc, handles
errors in the loggers (!).

> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> default to the empty list, and could be set to '(syslogd) in this case.
> 
> Does that sound too obscure to you, or would it be OK?

Sure, let's do that.

It's a little weird to have it for all agettys, although maybe some other users
of agetty require it anyway.

Then I wonder if all guix shepherd service configs should have such a field.

[1] https://isotope11.com/blog/manage-your-services-with-daemontools
[2] https://cr.yp.to/daemontools/faq/create.html
[3] https://cgit.freedesktop.org/systemd/systemd/tree/src/journal/journald-kmsg.c
dev_kmsg_fd = open("/dev/kmsg", O_RDWR|O_CLOEXEC|O_NONBLOCK|O_NOCTTY);
printf "<%i>" priority identifier "[" pid "]: " message "\n"

Linux:

	/*
	 * Extract and skip the syslog prefix <[0-9]*>. Coming from userspace
	 * the decimal value represents 32bit, the lower 3 bit are the log
	 * level, the rest are the log facility.
	 *
	 * If no prefix or no userspace facility is specified, we
	 * enforce LOG_USER [which is 1], to be able to reliably distinguish
	 * kernel-generated messages from userspace-injected ones.
	 */

[bug#30464] shepherd logging

[Danny Milosavljevic]

Oops, didn't see /var/log/shepherd.log - but I think it should just
use syslog for that - and it's possible to use syslog for that by
writing to /dev/kmsg .

Does the Hurd with /dev/kmsg ? How do they solve early boot logging ?

[bug#30464] shepherd logging; console-agetty-service

[Ludovic Courtès]

Hi,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Thu, 15 Feb 2018 16:47:54 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> (Which has me thinking that longer term it’d be nice to have the
>> Shepherd take care of syslogd-ish activity.)
>
> If you mean that we should make sure to log shepherd's own messages, I agree.

That’s already done.

> Integrating syslogd into shepherd, not sure.  I think external syslogd is fine.
> If not, there are a lot of other logging programs to choose from.
> I like modularity.

Yes, I like it too.  OTOH, there’s a chicken-and-egg problem here:
shepherd ends up implementing its own logging facility, and we have
situations like the one you mentioned earlier in this thread.

> shepherd could write its messages to the kernel log ringbuffer in /dev/kmsg [3].
> That sounds dirty, but it would synchronize messages oh-so-nicely and would
> not immediately require syslogd.  It would also make sure that syslogd
> eventually picks shepherd's messages up (right now they are somewhere on the
> first terminal - if you are lucky and they didn't scroll off).

Indeed, that’s something we can easily do already, and it would address
a major annoyance.  :-)  Actually, could it use syslog(3), which writes
to /dev/log?

> Also a way of capturing stderr and stdout (and maybe even /dev/log) of services
> would be nice.

Yes.  Though again capturing service stdout/stderr is kinda redundant
with what syslogd does.  What I like in journald is the fact that it
unifies all logging facilities, and also connects them to service
management.

> We could also instead open /dev/klog and dup2 its fd to 1 and 2.
> That way, shepherd messages and all stray messages by any process shepherd
> started will end up in the kernel log.  (problem: there are some reserved
> patterns that have special meaning - and we don't control what the services
> do as well as we do what just shepherd does)

Right, so we’d need shepherd to filter these and pass them through
syslog(3), which ensures correct message formatting.  That’s what
<https://cgit.freedesktop.org/systemd/systemd/tree/src/journal/journald-kmsg.c>
does apparently (thanks for the link!).

> Also, I know one is supposed to write UNIX services as daemons, but that's
> not really composable and kinda complicated to debug for no good reason.
> I'd prefer if shepherd also keeps a way to run regular programs as services,
> making sure that they are session leader, their output is logged, they are
> kept alive and monitored.

Sure!

> daemontools[1][2] have done all this stuff already and I like it much more
> than traditional service managers.  It's much more modular, handles logging
> on its own, handles error cases well, uses the file system well etc, handles
> errors in the loggers (!).

Sounds like a great source of inspiration then.  :-)

>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>> default to the empty list, and could be set to '(syslogd) in this case.
>> 
>> Does that sound too obscure to you, or would it be OK?
>
> Sure, let's do that.
>
> It's a little weird to have it for all agettys, although maybe some other users
> of agetty require it anyway.
>
> Then I wonder if all guix shepherd service configs should have such a field.

That brings us to the topic of a general service customization
mechanism: <https://bugs.gnu.org/27155>.  Well, one thing at a time.
:-)

Thanks for the great ideas!

Ludo’.

[bug#30464] shepherd logging

[Ludovic Courtès]

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> Oops, didn't see /var/log/shepherd.log - but I think it should just
> use syslog for that - and it's possible to use syslog for that by
> writing to /dev/kmsg .
>
> Does the Hurd with /dev/kmsg ?

It doesn’t have /dev/kmsg, but it has /dev/klog:

--8<---------------cut here---------------start------------->8---
ludo@darnassus:~$ ls -l /dev/klog 
crw-rw---- 1 root root 0, 0 Jul 16  2015 /dev/klog
ludo@darnassus:~$ showtrans /dev/klog 
/hurd/streamio kmsg
--8<---------------cut here---------------end--------------->8---

> How do they solve early boot logging ?

GNU Mach has a special “console” device that early boot processes can
open and write too, as in:

  https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731

It’s comparable to /dev/kmsg, but it exists before file system servers
are brought up, which is why it doesn’t have an entry in the file
system hierarchy.

Ludo’.

[bug#30464] shepherd logging

[Danny Milosavljevic]

>   https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731
> 
> It’s comparable to /dev/kmsg, but it exists before file system servers
> are brought up, which is why it doesn’t have an entry in the file
> system hierarchy.

Huh, so it's just connected to stderr?  That means it should work fine already
with no changes in shepherd.  Nice!

Just to be sure, it is possible to read out these messages and log them to a
file later on, right?

[bug#30464] shepherd logging

[Ludovic Courtès]

Danny Milosavljevic <dannym@scratchpost.org> skribis:

>>   https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731
>> 
>> It’s comparable to /dev/kmsg, but it exists before file system servers
>> are brought up, which is why it doesn’t have an entry in the file
>> system hierarchy.
>
> Huh, so it's just connected to stderr?  That means it should work fine already
> with no changes in shepherd.  Nice!

Indeed!

> Just to be sure, it is possible to read out these messages and log them to a
> file later on, right?

I think what goes to “console” is printed and then forever lost:

  https://git.savannah.gnu.org/cgit/hurd/gnumach.git/tree/device/cons.c

That said, not much should go there since as soon as the root file
system is up, processes can start using /dev/klog.

Ludo’.

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Ludovic Courtès]

Hi Danny,

The discussion drifted a bit…  :-)

ludo@gnu.org (Ludovic Courtès) skribis:

> Danny Milosavljevic <dannym@scratchpost.org> skribis:

[...]

>> It doesn't make sense to have agetty itself depend on syslogd because
>> agetty can be (and usually is) used for modems etc which have nothing
>> to do with the local console.
>
> Alright.
>
> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> default to the empty list, and could be set to '(syslogd) in this case.
>
> Does that sound too obscure to you, or would it be OK?

Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
this immediate need, I think.

Ludo’.

[bug#30464] [PATCH 1/2] services: Add console-agetty-service.

[Maxim Cournoyer]

Hello Danny,

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Danny,
>
> The discussion drifted a bit…  :-)
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
> [...]
>
>>> It doesn't make sense to have agetty itself depend on syslogd because
>>> agetty can be (and usually is) used for modems etc which have nothing
>>> to do with the local console.
>>
>> Alright.
>>
>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>> default to the empty list, and could be set to '(syslogd) in this case.
>>
>> Does that sound too obscure to you, or would it be OK?
>
> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
> this immediate need, I think.
>
> Ludo’.

Any update on this?

Maxim

[bug#30464] [PATCH 0/2] Add console-agetty-service.

[zimoun]

Hi,

This patch #30464 series [1] had been submitted on Feb 2018 and after a
drifting discussion, the last message is:


On Thu, 08 Oct 2020 at 14:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

>> The discussion drifted a bit…  :-)
>>
>> [...]
>>
>>>> It doesn't make sense to have agetty itself depend on syslogd because
>>>> agetty can be (and usually is) used for modems etc which have nothing
>>>> to do with the local console.
>>>
>>> Alright.
>>>
>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>>> default to the empty list, and could be set to '(syslogd) in this case.
>>>
>>> Does that sound too obscure to you, or would it be OK?
>>
>> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
>> this immediate need, I think.
>
> Any update on this?

Therefore, any update on this?


Cheers,
simon

1: <http://issues.guix.gnu.org/issue/30464>

[bug#30464] [PATCH 0/2] Add console-agetty-service.

[zimoun]

Hi,

For reference <http://issues.guix.gnu.org/issue/30464> from 2018.


On Thu, 13 Jan 2022 at 17:02, zimoun <zimon.toutoune@gmail.com> wrote:

> This patch #30464 series [1] had been submitted on Feb 2018 and after a
> drifting discussion, the last message is:
>
>
> On Thu, 08 Oct 2020 at 14:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>> The discussion drifted a bit…  :-)
>>>
>>> [...]
>>>
>>>>> It doesn't make sense to have agetty itself depend on syslogd because
>>>>> agetty can be (and usually is) used for modems etc which have nothing
>>>>> to do with the local console.
>>>>
>>>> Alright.
>>>>
>>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>>>> default to the empty list, and could be set to '(syslogd) in this case.
>>>>
>>>> Does that sound too obscure to you, or would it be OK?
>>>
>>> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
>>> this immediate need, I think.
>>
>> Any update on this?
>
> Therefore, any update on this?

Without any update on this, I plan to mark it as ’donewontfix’ in a
couple of weeks.  Let me know if it is worth to keep it open.


Cheers,
simon

[bug#30464] [PATCH 0/2] Add console-agetty-service.

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1243 bytes --]

Heh.

This problem is really not that bad in the big scheme of things.

I was hoping https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27155 to be done
first, obsoleting the need for this workaround here :(

> >>>>> It doesn't make sense to have agetty itself depend on syslogd because
> >>>>> agetty can be (and usually is) used for modems etc which have nothing
> >>>>> to do with the local console.  

For completeness:

It would also be possible to depend on syslogd by default and *also* have the
modem agettys startup delayed until syslogd is there.  But if you have a
modem agetty service but NO syslogd shepherd service in your config, that
would hang the boot with no remote access. So let's not do that.

27155 could maybe statically analyze whether there is a syslogd and an agetty
in the config, and error out if not both or neither.

> >>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> >>>> default to the empty list, and could be set to '(syslogd) in this case.

Yeah, we have to default to '() in the interest of backward compatibility
anyway.

> >>>> Does that sound too obscure to you, or would it be OK?  

Yes, it should be ok.

See bug# 53763 for a patchset.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[bug#30464] [PATCH 0/2] Add console-agetty-service.

[zimoun]

Hi Danny,

On Thu, 03 Feb 2022 at 21:55, Danny Milosavljevic <dannym@scratchpost.org> wrote:

> See bug# 53763 for a patchset.

Thanks!  Really cool that this old report is now closed. :-)


Cheers,
simon