all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* [bug#30464] [PATCH 0/2] Add console-agetty-service.
@ 2018-02-15 11:42 Danny Milosavljevic
  2018-02-15 11:47 ` [bug#30464] [PATCH 1/2] services: " Danny Milosavljevic
  0 siblings, 1 reply; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-15 11:42 UTC (permalink / raw)
  To: 30464

Danny Milosavljevic (2):
  services: Add console-agetty-service.
  services: agetty-shepherd-service: Default to providing  'term-console
    instead of 'term-auto .

 gnu/services/base.scm | 38 ++++++++++++++++++++++++++++++++------
 1 file changed, 32 insertions(+), 6 deletions(-)

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-15 11:42 [bug#30464] [PATCH 0/2] Add console-agetty-service Danny Milosavljevic
@ 2018-02-15 11:47 ` Danny Milosavljevic
  2018-02-15 11:47   ` [bug#30464] [PATCH 2/2] services: agetty-shepherd-service: Default to providing 'term-console instead of 'term-auto Danny Milosavljevic
  2018-02-15 14:40   ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
  0 siblings, 2 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-15 11:47 UTC (permalink / raw)
  To: 30464

* gnu/services/base.scm (<agetty-configuration>): Make tty default to #f.
(console-agetty-shepherd-service): New variable.
(console-agetty-service-type): New variable.
(console-agetty-service): New variable.
(%base-services): Replace agetty-service instance by console-agetty-service
instance.
---
 gnu/services/base.scm | 35 ++++++++++++++++++++++++++++++-----
 1 file changed, 30 insertions(+), 5 deletions(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 1ad7c2320..29ae5fca3 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -817,7 +817,8 @@ the message of the day, among other things."
   agetty-configuration?
   (agetty           agetty-configuration-agetty   ;<package>
                     (default util-linux))
-  (tty              agetty-configuration-tty)     ;string | #f
+  (tty              agetty-configuration-tty      ;string | #f
+                    (default #f))
   (term             agetty-term                   ;string | #f
                     (default #f))
   (baud-rate        agetty-baud-rate              ;string | #f
@@ -1072,6 +1073,31 @@ program.")))
 the tty to run, among other things."
   (service agetty-service-type config))
 
+(define console-agetty-shepherd-service
+  (lambda (config)
+    (let ((base-service
+           (car
+            (agetty-shepherd-service
+             (agetty-configuration (inherit config)
+              (keep-baud? #t))))))
+      (list
+       (shepherd-service (inherit base-service)
+         ;; Make sure that Linux doesn't log to the same console as well.
+         (requirement (cons 'syslogd (shepherd-service-requirement base-service))))))))
+
+(define console-agetty-service-type
+  (service-type (name 'console-agetty)
+                (extensions (list (service-extension shepherd-root-service-type
+                                                     console-agetty-shepherd-service)))
+                (description
+                 "Provide console login using the @command{agetty}
+program on the Linux console.")))
+
+(define* (console-agetty-service config)
+  "Return a service to run agetty according to @var{config}, which specifies
+the tty to run, defaulting to the Linux console."
+  (service console-agetty-service-type config))
+
 (define-record-type* <mingetty-configuration>
   mingetty-configuration make-mingetty-configuration
   mingetty-configuration?
@@ -2053,10 +2079,9 @@ This service is not part of @var{%base-services}."
                         (cons tty %default-console-font))
                       '("tty1" "tty2" "tty3" "tty4" "tty5" "tty6")))
 
-        (agetty-service (agetty-configuration
-                         (extra-options '("-L")) ; no carrier detect
-                         (term "vt100")
-                         (tty #f))) ; automatic
+        (console-agetty-service (agetty-configuration
+                                 (extra-options '("-L")) ; no carrier detect
+                                 (term "vt100")))
 
         (mingetty-service (mingetty-configuration
                            (tty "tty1")))

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 2/2] services: agetty-shepherd-service: Default to providing 'term-console instead of 'term-auto .
  2018-02-15 11:47 ` [bug#30464] [PATCH 1/2] services: " Danny Milosavljevic
@ 2018-02-15 11:47   ` Danny Milosavljevic
  2018-02-15 14:40   ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
  1 sibling, 0 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-15 11:47 UTC (permalink / raw)
  To: 30464

* gnu/services/base.scm (agetty-shepherd-service): Default to providing
'term-console instead of 'term-auto .
---
 gnu/services/base.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 29ae5fca3..4ab83d1d2 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -937,7 +937,8 @@ to use as the tty.  This is primarily useful for headless systems."
        (shepherd-service
          (modules '((ice-9 match) (gnu build linux-boot)))
          (documentation "Run agetty on a tty.")
-         (provision (list (symbol-append 'term- (string->symbol (or tty "auto")))))
+         (provision (list (symbol-append 'term- (string->symbol
+                                                 (or tty "console")))))
 
          ;; Since the login prompt shows the host name, wait for the 'host-name'
          ;; service to be done.  Also wait for udev essentially so that the tty

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-15 11:47 ` [bug#30464] [PATCH 1/2] services: " Danny Milosavljevic
  2018-02-15 11:47   ` [bug#30464] [PATCH 2/2] services: agetty-shepherd-service: Default to providing 'term-console instead of 'term-auto Danny Milosavljevic
@ 2018-02-15 14:40   ` Ludovic Courtès
  2018-02-15 15:41     ` Danny Milosavljevic
  1 sibling, 1 reply; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-15 14:40 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Hello,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> * gnu/services/base.scm (<agetty-configuration>): Make tty default to #f.
> (console-agetty-shepherd-service): New variable.
> (console-agetty-service-type): New variable.
> (console-agetty-service): New variable.
> (%base-services): Replace agetty-service instance by console-agetty-service
> instance.

Could you explain the rationale?

The only difference is that this shepherd service depends on ‘syslogd’
whereas the current agetty service doesn’t, right?

> +(define* (console-agetty-service config)
> +  "Return a service to run agetty according to @var{config}, which specifies
> +the tty to run, defaulting to the Linux console."
> +  (service console-agetty-service-type config))

In general we don’t provide such procedures any more and instead expose
service types and configs directly.

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-15 14:40   ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
@ 2018-02-15 15:41     ` Danny Milosavljevic
  2018-02-15 15:47       ` Ludovic Courtès
  0 siblings, 1 reply; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-15 15:41 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30464

Hi Ludo,

On Thu, 15 Feb 2018 15:40:14 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Could you explain the rationale?
> 
> The only difference is that this shepherd service depends on ‘syslogd’
> whereas the current agetty service doesn’t, right?

Yes, it's my inner OCD.

Right now, the agetty is started while the Linux kernel is still booting and
printing stuff to the console.  So you have two parties printing stuff
to the same console simultaneously.  That looks ... not nice.

What this would do is the getty would only be started (and printing stuff)
after syslogd.  The first thing syslogd does is it disables the Linux direct
console printing.

It doesn't make sense to have agetty itself depend on syslogd because
agetty can be (and usually is) used for modems etc which have nothing
to do with the local console.

> > +(define* (console-agetty-service config)
> > +  "Return a service to run agetty according to @var{config}, which specifies
> > +the tty to run, defaulting to the Linux console."
> > +  (service console-agetty-service-type config))  
> 
> In general we don’t provide such procedures any more and instead expose
> service types and configs directly.

Okay, I can remove that one and inline it at the (only) call site.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-15 15:41     ` Danny Milosavljevic
@ 2018-02-15 15:47       ` Ludovic Courtès
  2018-02-16 20:57         ` [bug#30464] shepherd logging; console-agetty-service Danny Milosavljevic
  2018-02-26 17:52         ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
  0 siblings, 2 replies; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-15 15:47 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Heya,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Thu, 15 Feb 2018 15:40:14 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Could you explain the rationale?
>> 
>> The only difference is that this shepherd service depends on ‘syslogd’
>> whereas the current agetty service doesn’t, right?
>
> Yes, it's my inner OCD.
>
> Right now, the agetty is started while the Linux kernel is still booting and
> printing stuff to the console.  So you have two parties printing stuff
> to the same console simultaneously.  That looks ... not nice.

Right.

> What this would do is the getty would only be started (and printing stuff)
> after syslogd.  The first thing syslogd does is it disables the Linux direct
> console printing.

OK, got it.

(Which has me thinking that longer term it’d be nice to have the
Shepherd take care of syslogd-ish activity.)

> It doesn't make sense to have agetty itself depend on syslogd because
> agetty can be (and usually is) used for modems etc which have nothing
> to do with the local console.

Alright.

How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
default to the empty list, and could be set to '(syslogd) in this case.

Does that sound too obscure to you, or would it be OK?

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging; console-agetty-service
  2018-02-15 15:47       ` Ludovic Courtès
@ 2018-02-16 20:57         ` Danny Milosavljevic
  2018-02-16 21:07           ` [bug#30464] shepherd logging Danny Milosavljevic
  2018-02-17 16:20           ` [bug#30464] shepherd logging; console-agetty-service Ludovic Courtès
  2018-02-26 17:52         ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
  1 sibling, 2 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-16 20:57 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30464

Hi Ludo,

On Thu, 15 Feb 2018 16:47:54 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> (Which has me thinking that longer term it’d be nice to have the
> Shepherd take care of syslogd-ish activity.)

If you mean that we should make sure to log shepherd's own messages, I agree.

Integrating syslogd into shepherd, not sure.  I think external syslogd is fine.
If not, there are a lot of other logging programs to choose from.
I like modularity.

shepherd could write its messages to the kernel log ringbuffer in /dev/kmsg [3].
That sounds dirty, but it would synchronize messages oh-so-nicely and would
not immediately require syslogd.  It would also make sure that syslogd
eventually picks shepherd's messages up (right now they are somewhere on the
first terminal - if you are lucky and they didn't scroll off).

I'm not sure whether then they would be printed to /dev/console as well then -
probably.

We'd need a guile soft-port, but it's not like I haven't done that before.

User-shepherd shouldn't do it though (and can't because it doesn't have
permission to write to /dev/kmsg).

Please stop me and tell me why it's a bad idea :)

Also a way of capturing stderr and stdout (and maybe even /dev/log) of services
would be nice.

If you thought that the above was bad, you ain't seen nothing yet :->

We could also instead open /dev/klog and dup2 its fd to 1 and 2.
That way, shepherd messages and all stray messages by any process shepherd
started will end up in the kernel log.  (problem: there are some reserved
patterns that have special meaning - and we don't control what the services
do as well as we do what just shepherd does)

Also, I know one is supposed to write UNIX services as daemons, but that's
not really composable and kinda complicated to debug for no good reason.
I'd prefer if shepherd also keeps a way to run regular programs as services,
making sure that they are session leader, their output is logged, they are
kept alive and monitored.

daemontools[1][2] have done all this stuff already and I like it much more
than traditional service managers.  It's much more modular, handles logging
on its own, handles error cases well, uses the file system well etc, handles
errors in the loggers (!).

> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> default to the empty list, and could be set to '(syslogd) in this case.
> 
> Does that sound too obscure to you, or would it be OK?

Sure, let's do that.

It's a little weird to have it for all agettys, although maybe some other users
of agetty require it anyway.

Then I wonder if all guix shepherd service configs should have such a field.

[1] https://isotope11.com/blog/manage-your-services-with-daemontools
[2] https://cr.yp.to/daemontools/faq/create.html
[3] https://cgit.freedesktop.org/systemd/systemd/tree/src/journal/journald-kmsg.c
dev_kmsg_fd = open("/dev/kmsg", O_RDWR|O_CLOEXEC|O_NONBLOCK|O_NOCTTY);
printf "<%i>" priority identifier "[" pid "]: " message "\n"

Linux:

	/*
	 * Extract and skip the syslog prefix <[0-9]*>. Coming from userspace
	 * the decimal value represents 32bit, the lower 3 bit are the log
	 * level, the rest are the log facility.
	 *
	 * If no prefix or no userspace facility is specified, we
	 * enforce LOG_USER [which is 1], to be able to reliably distinguish
	 * kernel-generated messages from userspace-injected ones.
	 */

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging
  2018-02-16 20:57         ` [bug#30464] shepherd logging; console-agetty-service Danny Milosavljevic
@ 2018-02-16 21:07           ` Danny Milosavljevic
  2018-02-16 23:17             ` Danny Milosavljevic
  2018-02-17 16:25             ` [bug#30464] " Ludovic Courtès
  2018-02-17 16:20           ` [bug#30464] shepherd logging; console-agetty-service Ludovic Courtès
  1 sibling, 2 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-16 21:07 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30464

Oops, didn't see /var/log/shepherd.log - but I think it should just
use syslog for that - and it's possible to use syslog for that by
writing to /dev/kmsg .

Does the Hurd with /dev/kmsg ? How do they solve early boot logging ?

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: shepherd logging
  2018-02-16 21:07           ` [bug#30464] shepherd logging Danny Milosavljevic
@ 2018-02-16 23:17             ` Danny Milosavljevic
  2018-02-17 16:25             ` [bug#30464] " Ludovic Courtès
  1 sibling, 0 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-16 23:17 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

Aaand it works on Linux - shepherd messages show up on the console and eventually
end up in syslog (I checked).  Integration is surprisingly easy and safe (shepherd
already does line buffering manually):

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index ccf62a6e2..168495c14 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -169,7 +169,15 @@ and provides a \"top-like\" mode (monitoring).")
               (patches (search-patches "shepherd-close-fds.patch"))))
     (build-system gnu-build-system)
     (arguments
-     '(#:configure-flags '("--localstatedir=/var")))
+     '(#:configure-flags '("--localstatedir=/var")
+       #:tests? #f ; I broke them, sorry
+       #:phases
+       (modify-phases %standard-phases
+        (add-after 'unpack 'patch-kmsg
+          (lambda _
+            (substitute* "modules/shepherd/support.scm"
+             (("%localstatedir \"/log/shepherd.log\"") "\"/dev/kmsg\""))
+            #t)))))
     (native-inputs
      `(("pkg-config" ,pkg-config)

Timestamp is printed twice, though (shepherd prints one, too).

$ cat /var/log/messages
[...]
Feb 16 23:01:58 localhost vmunix: [    9.178841] Console: switching to colour frame buffer device 160x50
Feb 16 23:01:58 localhost vmunix: [    9.198267] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
Feb 16 23:01:58 localhost vmunix: [    9.229274] 2018-02-16 23:01:32 Service udev has been started. <----- hellooooo
Feb 16 23:01:58 localhost vmunix: [   25.232541] NET: Registered protocol family 38

Note: 9.229274 is [Fr Feb 16 23:01:32 2018] (as checked by "dmesg -T"),
so syslogd is kinda off here.  Maybe they print the time of their /dev/kmsg
readout action... that would be weird.

Maybe this would remove the duplicate timestamp:

diff --git a/modules/shepherd/comm.scm b/modules/shepherd/comm.scm
index 0228f63..4048315 100644
--- a/modules/shepherd/comm.scm
+++ b/modules/shepherd/comm.scm
@@ -51,6 +51,7 @@
             start-logging
             stop-logging
             %current-client-socket
+            %current-logfile-date-format
             shepherd-output-port))
 
 ^L
@@ -200,6 +201,9 @@ on service '~a':")
   ;; Socket of the client currently talking to the daemon.
   (make-parameter #f))
 
+(define %current-logfile-date-format
+  (make-parameter default-logfile-date-format))
+
 ;; We provide our own output mechanism, because we have certain
 ;; special needs; most importantly, we want to send output to herd
 ;; sometimes.
@@ -228,7 +232,7 @@ on service '~a':")
             (let* ((log (lambda (x)
                           (display x log-output-port)))
                    (init-line (lambda ()
-                                (log (strftime "%Y-%m-%d %H:%M:%S "
+                                (log (strftime %current-logfile-date-format
                                                (localtime (current-time)))))))
               (init-line)
               (for-each log (reverse buffer))
diff --git a/modules/shepherd/support.scm b/modules/shepherd/support.scm
index bb01edc..0df97ea 100644
--- a/modules/shepherd/support.scm
+++ b/modules/shepherd/support.scm
@@ -285,6 +285,11 @@ TARGET should be a string representing a filepath + name."
       (string-append %localstatedir "/log/shepherd.log")
       (string-append %user-config-dir "/shepherd.log")))
 
+(define default-logfile-date-format
+   (if (zero? (getuid))
+       (format #f "[~d]: " (getpid))
+       "%Y-%m-%d %H:%M:%S "))
+
 ;; Configuration file.
 (define (default-config-file)
   "Return the default configuration file---either the user's file, or the

Doesn't work on the Hurd (even cat /dev/klog hangs the Hurd).

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging; console-agetty-service
  2018-02-16 20:57         ` [bug#30464] shepherd logging; console-agetty-service Danny Milosavljevic
  2018-02-16 21:07           ` [bug#30464] shepherd logging Danny Milosavljevic
@ 2018-02-17 16:20           ` Ludovic Courtès
  2018-02-17 18:02             ` Danny Milosavljevic
  1 sibling, 1 reply; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-17 16:20 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Hi,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Thu, 15 Feb 2018 16:47:54 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
>> (Which has me thinking that longer term it’d be nice to have the
>> Shepherd take care of syslogd-ish activity.)
>
> If you mean that we should make sure to log shepherd's own messages, I agree.

That’s already done.

> Integrating syslogd into shepherd, not sure.  I think external syslogd is fine.
> If not, there are a lot of other logging programs to choose from.
> I like modularity.

Yes, I like it too.  OTOH, there’s a chicken-and-egg problem here:
shepherd ends up implementing its own logging facility, and we have
situations like the one you mentioned earlier in this thread.

> shepherd could write its messages to the kernel log ringbuffer in /dev/kmsg [3].
> That sounds dirty, but it would synchronize messages oh-so-nicely and would
> not immediately require syslogd.  It would also make sure that syslogd
> eventually picks shepherd's messages up (right now they are somewhere on the
> first terminal - if you are lucky and they didn't scroll off).

Indeed, that’s something we can easily do already, and it would address
a major annoyance.  :-)  Actually, could it use syslog(3), which writes
to /dev/log?

> Also a way of capturing stderr and stdout (and maybe even /dev/log) of services
> would be nice.

Yes.  Though again capturing service stdout/stderr is kinda redundant
with what syslogd does.  What I like in journald is the fact that it
unifies all logging facilities, and also connects them to service
management.

> We could also instead open /dev/klog and dup2 its fd to 1 and 2.
> That way, shepherd messages and all stray messages by any process shepherd
> started will end up in the kernel log.  (problem: there are some reserved
> patterns that have special meaning - and we don't control what the services
> do as well as we do what just shepherd does)

Right, so we’d need shepherd to filter these and pass them through
syslog(3), which ensures correct message formatting.  That’s what
<https://cgit.freedesktop.org/systemd/systemd/tree/src/journal/journald-kmsg.c>
does apparently (thanks for the link!).

> Also, I know one is supposed to write UNIX services as daemons, but that's
> not really composable and kinda complicated to debug for no good reason.
> I'd prefer if shepherd also keeps a way to run regular programs as services,
> making sure that they are session leader, their output is logged, they are
> kept alive and monitored.

Sure!

> daemontools[1][2] have done all this stuff already and I like it much more
> than traditional service managers.  It's much more modular, handles logging
> on its own, handles error cases well, uses the file system well etc, handles
> errors in the loggers (!).

Sounds like a great source of inspiration then.  :-)

>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>> default to the empty list, and could be set to '(syslogd) in this case.
>> 
>> Does that sound too obscure to you, or would it be OK?
>
> Sure, let's do that.
>
> It's a little weird to have it for all agettys, although maybe some other users
> of agetty require it anyway.
>
> Then I wonder if all guix shepherd service configs should have such a field.

That brings us to the topic of a general service customization
mechanism: <https://bugs.gnu.org/27155>.  Well, one thing at a time.
:-)

Thanks for the great ideas!

Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging
  2018-02-16 21:07           ` [bug#30464] shepherd logging Danny Milosavljevic
  2018-02-16 23:17             ` Danny Milosavljevic
@ 2018-02-17 16:25             ` Ludovic Courtès
  2018-02-17 16:47               ` Danny Milosavljevic
  1 sibling, 1 reply; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-17 16:25 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> Oops, didn't see /var/log/shepherd.log - but I think it should just
> use syslog for that - and it's possible to use syslog for that by
> writing to /dev/kmsg .
>
> Does the Hurd with /dev/kmsg ?

It doesn’t have /dev/kmsg, but it has /dev/klog:

--8<---------------cut here---------------start------------->8---
ludo@darnassus:~$ ls -l /dev/klog 
crw-rw---- 1 root root 0, 0 Jul 16  2015 /dev/klog
ludo@darnassus:~$ showtrans /dev/klog 
/hurd/streamio kmsg
--8<---------------cut here---------------end--------------->8---

> How do they solve early boot logging ?

GNU Mach has a special “console” device that early boot processes can
open and write too, as in:

  https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731

It’s comparable to /dev/kmsg, but it exists before file system servers
are brought up, which is why it doesn’t have an entry in the file
system hierarchy.

Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging
  2018-02-17 16:25             ` [bug#30464] " Ludovic Courtès
@ 2018-02-17 16:47               ` Danny Milosavljevic
  2018-02-17 16:57                 ` Ludovic Courtès
  0 siblings, 1 reply; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-17 16:47 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 30464

>   https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731
> 
> It’s comparable to /dev/kmsg, but it exists before file system servers
> are brought up, which is why it doesn’t have an entry in the file
> system hierarchy.

Huh, so it's just connected to stderr?  That means it should work fine already
with no changes in shepherd.  Nice!

Just to be sure, it is possible to read out these messages and log them to a
file later on, right?

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] shepherd logging
  2018-02-17 16:47               ` Danny Milosavljevic
@ 2018-02-17 16:57                 ` Ludovic Courtès
  0 siblings, 0 replies; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-17 16:57 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Danny Milosavljevic <dannym@scratchpost.org> skribis:

>>   https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/startup/startup.c#n731
>> 
>> It’s comparable to /dev/kmsg, but it exists before file system servers
>> are brought up, which is why it doesn’t have an entry in the file
>> system hierarchy.
>
> Huh, so it's just connected to stderr?  That means it should work fine already
> with no changes in shepherd.  Nice!

Indeed!

> Just to be sure, it is possible to read out these messages and log them to a
> file later on, right?

I think what goes to “console” is printed and then forever lost:

  https://git.savannah.gnu.org/cgit/hurd/gnumach.git/tree/device/cons.c

That said, not much should go there since as soon as the root file
system is up, processes can start using /dev/klog.

Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: shepherd logging; console-agetty-service
  2018-02-17 16:20           ` [bug#30464] shepherd logging; console-agetty-service Ludovic Courtès
@ 2018-02-17 18:02             ` Danny Milosavljevic
  0 siblings, 0 replies; 20+ messages in thread
From: Danny Milosavljevic @ 2018-02-17 18:02 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

Hi Ludo,

On Sat, 17 Feb 2018 17:20:06 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> Actually, could [shepherd] use syslog(3), which writes to /dev/log?

As far as I understand the meaning overload in Linux, there's these:

* syslog(2) - accesses the kernel log ringbuffer directly.
glibc (rightly) pretends this doesn't exist.
* /dev/kmsg - userspace access to read and write to the kernel
log ringbuffer.  It doesn't really work like a regular file.
If you write(2) to it, that gets interpreted as a *line*
of max length (1024 - 32) and this line gets added to the
ringbuffer as a new message.
Reading is weird too - but we won't use that directly.
* syslog(3) - actually writes to /dev/log, a socket, which is
usually listened to by a user-space process like syslogd.
No kernel involved anywhere (well, I guess the BSD socket
layer is involved...).

If shepherd used /dev/log in the beginning that would mean that
syslogd would have to be running before shepherd starts up.

So syslogd would have pid 1, I guess ;)

If we instead used /dev/kmsg, there would not really be a hard dependency.
Linux takes care that the messages are printed to the console
until a syslogd (maybe) appears, at which point syslogd will
log the already-printed (to /dev/kmsg) messages to its output files.

syslogd also fetches LATER messages from /dev/kmsg and
appends them to its output files likewise as time goes by.

So the UNIXy good-enough way would be for us to only use /dev/kmsg, the end :)

The complete way (which would avoid a hypothetical problem of overloading
the ringbuffer with messages so we would lose some of them) would be
to have shepherd start out using /dev/kmsg and switch over to using /dev/log
when it's there (maybe use inotify and watch until /dev/log appears).
Switch back to /dev/kmsg if /dev/log breaks again.

Also possible would be to have shepherd provide /dev/log to user services
and have shepherd proxy them to /dev/kmsg - that sounds like a real easy way
to get into a cycle if we aren't careful - although it would be conceptually
nicer.

In fact, the Linux kernel could have done that transparently.  They probably
just didn't like the failover handling involved :)

Also, I checked glibc misc/syslog.c vsyslog and they fall back to
/dev/console - disappointing.
They could have fallen back to /dev/kmsg ...

But then if syslogd crashed nobody would re-enable console logging and it wouldn't
be seen until syslogd started up again...  it's actually starting to get
complicated again :)

> > Also a way of capturing stderr and stdout (and maybe even /dev/log) of services
> > would be nice.  
> 
> Yes.  Though again capturing service stdout/stderr is kinda redundant
> with what syslogd does.  What I like in journald is the fact that it
> unifies all logging facilities, and also connects them to service
> management.

Yeah, its nice nowadays after one gets used to it.

In the end, it depends on whether one wants a system overview or a
service-by-service overview.

Older UNIX had lots of small programs working together in harmony to
achieve one task that the user cared about.

If you just watched the logs of one of them that wouldn't really help
that much because you'd not see the parts the other guys did.

And if you then watched *their* logs in an extra terminal or something
you've have to manually cobble together the timeline in your head to
make sense of what you are seeing.

Sometimes some of the services involved were on other machines.

So it was understandable that syslogd came to be a "global" thing
(sometimes on a central server only).

Nowadays with all the huge-ish services like dnsmasq which do
everything on their own (DNS, DHCP, TFTP, kitchensink) just checking
its log is enough.

Not sure how (or whether) to unify both use cases.

> Right, so we’d need shepherd to filter these and pass them through
> syslog(3), which ensures correct message formatting.  That’s what
> <https://cgit.freedesktop.org/systemd/systemd/tree/src/journal/journald-kmsg.c>
> does apparently (thanks for the link!).

Yeah.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-15 15:47       ` Ludovic Courtès
  2018-02-16 20:57         ` [bug#30464] shepherd logging; console-agetty-service Danny Milosavljevic
@ 2018-02-26 17:52         ` Ludovic Courtès
  2020-10-08 18:21           ` Maxim Cournoyer
  1 sibling, 1 reply; 20+ messages in thread
From: Ludovic Courtès @ 2018-02-26 17:52 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: 30464

Hi Danny,

The discussion drifted a bit…  :-)

ludo@gnu.org (Ludovic Courtès) skribis:

> Danny Milosavljevic <dannym@scratchpost.org> skribis:

[...]

>> It doesn't make sense to have agetty itself depend on syslogd because
>> agetty can be (and usually is) used for modems etc which have nothing
>> to do with the local console.
>
> Alright.
>
> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> default to the empty list, and could be set to '(syslogd) in this case.
>
> Does that sound too obscure to you, or would it be OK?

Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
this immediate need, I think.

Ludo’.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 1/2] services: Add console-agetty-service.
  2018-02-26 17:52         ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
@ 2020-10-08 18:21           ` Maxim Cournoyer
  2022-01-13 16:02             ` [bug#30464] [PATCH 0/2] " zimoun
  0 siblings, 1 reply; 20+ messages in thread
From: Maxim Cournoyer @ 2020-10-08 18:21 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: Danny Milosavljevic, 30464

Hello Danny,

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Danny,
>
> The discussion drifted a bit…  :-)
>
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> Danny Milosavljevic <dannym@scratchpost.org> skribis:
>
> [...]
>
>>> It doesn't make sense to have agetty itself depend on syslogd because
>>> agetty can be (and usually is) used for modems etc which have nothing
>>> to do with the local console.
>>
>> Alright.
>>
>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>> default to the empty list, and could be set to '(syslogd) in this case.
>>
>> Does that sound too obscure to you, or would it be OK?
>
> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
> this immediate need, I think.
>
> Ludo’.

Any update on this?

Maxim




^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 0/2] Add console-agetty-service.
  2020-10-08 18:21           ` Maxim Cournoyer
@ 2022-01-13 16:02             ` zimoun
  2022-02-03  2:33               ` zimoun
  0 siblings, 1 reply; 20+ messages in thread
From: zimoun @ 2022-01-13 16:02 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: Danny Milosavljevic, Ludovic Courtès, 30464

Hi,

This patch #30464 series [1] had been submitted on Feb 2018 and after a
drifting discussion, the last message is:


On Thu, 08 Oct 2020 at 14:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

>> The discussion drifted a bit…  :-)
>>
>> [...]
>>
>>>> It doesn't make sense to have agetty itself depend on syslogd because
>>>> agetty can be (and usually is) used for modems etc which have nothing
>>>> to do with the local console.
>>>
>>> Alright.
>>>
>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>>> default to the empty list, and could be set to '(syslogd) in this case.
>>>
>>> Does that sound too obscure to you, or would it be OK?
>>
>> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
>> this immediate need, I think.
>
> Any update on this?

Therefore, any update on this?


Cheers,
simon

1: <http://issues.guix.gnu.org/issue/30464>




^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 0/2] Add console-agetty-service.
  2022-01-13 16:02             ` [bug#30464] [PATCH 0/2] " zimoun
@ 2022-02-03  2:33               ` zimoun
  2022-02-03 20:55                 ` Danny Milosavljevic
  0 siblings, 1 reply; 20+ messages in thread
From: zimoun @ 2022-02-03  2:33 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: Danny Milosavljevic, Ludovic Courtès, 30464

Hi,

For reference <http://issues.guix.gnu.org/issue/30464> from 2018.


On Thu, 13 Jan 2022 at 17:02, zimoun <zimon.toutoune@gmail.com> wrote:

> This patch #30464 series [1] had been submitted on Feb 2018 and after a
> drifting discussion, the last message is:
>
>
> On Thu, 08 Oct 2020 at 14:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>> The discussion drifted a bit…  :-)
>>>
>>> [...]
>>>
>>>>> It doesn't make sense to have agetty itself depend on syslogd because
>>>>> agetty can be (and usually is) used for modems etc which have nothing
>>>>> to do with the local console.
>>>>
>>>> Alright.
>>>>
>>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
>>>> default to the empty list, and could be set to '(syslogd) in this case.
>>>>
>>>> Does that sound too obscure to you, or would it be OK?
>>>
>>> Thoughts?  The ‘dependencies’ hack looks simple enough and would solve
>>> this immediate need, I think.
>>
>> Any update on this?
>
> Therefore, any update on this?

Without any update on this, I plan to mark it as ’donewontfix’ in a
couple of weeks.  Let me know if it is worth to keep it open.


Cheers,
simon




^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 0/2] Add console-agetty-service.
  2022-02-03  2:33               ` zimoun
@ 2022-02-03 20:55                 ` Danny Milosavljevic
  2022-03-23 13:00                   ` zimoun
  0 siblings, 1 reply; 20+ messages in thread
From: Danny Milosavljevic @ 2022-02-03 20:55 UTC (permalink / raw)
  To: zimoun; +Cc: Ludovic Courtès, 30464, Maxim Cournoyer

[-- Attachment #1: Type: text/plain, Size: 1243 bytes --]

Heh.

This problem is really not that bad in the big scheme of things.

I was hoping https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27155 to be done
first, obsoleting the need for this workaround here :(

> >>>>> It doesn't make sense to have agetty itself depend on syslogd because
> >>>>> agetty can be (and usually is) used for modems etc which have nothing
> >>>>> to do with the local console.  

For completeness:

It would also be possible to depend on syslogd by default and *also* have the
modem agettys startup delayed until syslogd is there.  But if you have a
modem agetty service but NO syslogd shepherd service in your config, that
would hang the boot with no remote access. So let's not do that.

27155 could maybe statically analyze whether there is a syslogd and an agetty
in the config, and error out if not both or neither.

> >>>> How about adding a ‘dependencies’ field to <agetty-configuration>?  It’d
> >>>> default to the empty list, and could be set to '(syslogd) in this case.

Yeah, we have to default to '() in the interest of backward compatibility
anyway.

> >>>> Does that sound too obscure to you, or would it be OK?  

Yes, it should be ok.

See bug# 53763 for a patchset.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [bug#30464] [PATCH 0/2] Add console-agetty-service.
  2022-02-03 20:55                 ` Danny Milosavljevic
@ 2022-03-23 13:00                   ` zimoun
  0 siblings, 0 replies; 20+ messages in thread
From: zimoun @ 2022-03-23 13:00 UTC (permalink / raw)
  To: Danny Milosavljevic; +Cc: Ludovic Courtès, 30464, Maxim Cournoyer

Hi Danny,

On Thu, 03 Feb 2022 at 21:55, Danny Milosavljevic <dannym@scratchpost.org> wrote:

> See bug# 53763 for a patchset.

Thanks!  Really cool that this old report is now closed. :-)


Cheers,
simon




^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2022-03-23 13:09 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-15 11:42 [bug#30464] [PATCH 0/2] Add console-agetty-service Danny Milosavljevic
2018-02-15 11:47 ` [bug#30464] [PATCH 1/2] services: " Danny Milosavljevic
2018-02-15 11:47   ` [bug#30464] [PATCH 2/2] services: agetty-shepherd-service: Default to providing 'term-console instead of 'term-auto Danny Milosavljevic
2018-02-15 14:40   ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
2018-02-15 15:41     ` Danny Milosavljevic
2018-02-15 15:47       ` Ludovic Courtès
2018-02-16 20:57         ` [bug#30464] shepherd logging; console-agetty-service Danny Milosavljevic
2018-02-16 21:07           ` [bug#30464] shepherd logging Danny Milosavljevic
2018-02-16 23:17             ` Danny Milosavljevic
2018-02-17 16:25             ` [bug#30464] " Ludovic Courtès
2018-02-17 16:47               ` Danny Milosavljevic
2018-02-17 16:57                 ` Ludovic Courtès
2018-02-17 16:20           ` [bug#30464] shepherd logging; console-agetty-service Ludovic Courtès
2018-02-17 18:02             ` Danny Milosavljevic
2018-02-26 17:52         ` [bug#30464] [PATCH 1/2] services: Add console-agetty-service Ludovic Courtès
2020-10-08 18:21           ` Maxim Cournoyer
2022-01-13 16:02             ` [bug#30464] [PATCH 0/2] " zimoun
2022-02-03  2:33               ` zimoun
2022-02-03 20:55                 ` Danny Milosavljevic
2022-03-23 13:00                   ` zimoun

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.