* Services and log management/monitoring @ 2019-04-14 18:12 rendaw 2019-04-17 21:20 ` Ludovic Courtès 0 siblings, 1 reply; 9+ messages in thread From: rendaw @ 2019-04-14 18:12 UTC (permalink / raw) To: help-guix The manual talks about standard services vs shepherd services, but doesn't really discuss which either are. My impression is a standard service is simply a process guix runs on the system (at startup, based on the graph?), where a shepherd service is a process that is managed (may be manually started/stopped, automatically restarted, is watched) by shepherd. Is that interpretation correct? My expectation would then be that you should generally use shepherd services, except when you want very fine control over how the service is managed. Do shepherd services automatically get enabled and start when configured in a guix config? Okay, just reread everything and I'm quite confused. It says shepherd is pid 1, so "shepherd" services are the more fundamental service type? But the manual says that shepherd services are a particular type of service, which implies the opposite. Why would one use "service" instead of "shepherd-service" (or vice-versa)? Is it possible to define a service which depends on a shepherd-service? What about the reverse? -- On logging - one of the things I like about systemd is that all services log their stdout/err and there's a standard way to retrieve logs, and especially query logs from a particular time range/boot. It also exposes dbus events when service statuses change - so it's pretty easy to send a notification if a service dies for instance, maybe with the logs from the 10m leading up to the issue. I couldn't find anything about logging and monitoring in the guix manual (other than statistic collecting agents/dashboards). In the shepherd manual all I could find was a note that services send stdout/stderr to syslog. Are any of the above possible with this setup? I've used syslog systems for years up until systemd, but my impression was that it was basically a pile of random files you can read manually if you really must. Do non-shepherd services also log somewhere? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-04-14 18:12 Services and log management/monitoring rendaw @ 2019-04-17 21:20 ` Ludovic Courtès 2019-04-18 15:11 ` rendaw 0 siblings, 1 reply; 9+ messages in thread From: Ludovic Courtès @ 2019-04-17 21:20 UTC (permalink / raw) To: rendaw; +Cc: help-guix Hello! The kind of “service” that Guix manages is those described here: https://www.gnu.org/software/guix/manual/en/html_node/Service-Composition.html The first paragraph tries to explain it: Here we define a “service” as, broadly, something that extends the functionality of the operating system. Often a service is a process—a “daemon”—started when the system boots: a secure shell server, a Web server, the Guix build daemon, etc. Sometimes a service is a daemon whose execution can be triggered by another daemon—e.g., an FTP server started by ‘inetd’ or a D-Bus service activated by ‘dbus-daemon’. Occasionally, a service does not map to a daemon. For instance, the “account” service collects user accounts and makes sure they exist when the system runs; the “udev” service collects device management rules and makes them available to the eudev daemon; the ‘/etc’ service populates the ‘/etc’ directory of the system. For services that “map to a daemon”, you’d extend ‘shepherd-root-service-type’ by providing a Shepherd service. A Shepherd service is a service managed by PID 1. You can list them on a running system by running ‘herd status’ as root. IOW, Shepherd services are a special case of service. Just like D-Bus services are another special case, etc. About logging: Shepherd does very little in that area. It limits itself to providing a #:log-file parameter to capture the processes’s stdout/stderr to a file. Other than that we usually configure daemons to write to syslog, which provides more flexibility regarding storage and filtering of log entries. A “non-Shepherd service” as you call them doesn’t necessarily map to a process, so there’s potentially nothing to log in the first place. Does that answer your questions? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-04-17 21:20 ` Ludovic Courtès @ 2019-04-18 15:11 ` rendaw 2019-04-19 12:09 ` Ludovic Courtès 0 siblings, 1 reply; 9+ messages in thread From: rendaw @ 2019-04-18 15:11 UTC (permalink / raw) To: Ludovic Courtès; +Cc: help-guix On 4/18/19 6:20 AM, Ludovic Courtès wrote: > Hello! > > The kind of “service” that Guix manages is those described here: > > https://www.gnu.org/software/guix/manual/en/html_node/Service-Composition.html > > The first paragraph tries to explain it: > > Here we define a “service” as, broadly, something that extends the > functionality of the operating system. Often a service is a process—a > “daemon”—started when the system boots: a secure shell server, a Web > server, the Guix build daemon, etc. Sometimes a service is a daemon > whose execution can be triggered by another daemon—e.g., an FTP server > started by ‘inetd’ or a D-Bus service activated by ‘dbus-daemon’. > Occasionally, a service does not map to a daemon. For instance, the > “account” service collects user accounts and makes sure they exist when > the system runs; the “udev” service collects device management rules and > makes them available to the eudev daemon; the ‘/etc’ service populates > the ‘/etc’ directory of the system. > > For services that “map to a daemon”, you’d extend > ‘shepherd-root-service-type’ by providing a Shepherd service. A > Shepherd service is a service managed by PID 1. You can list them on a > running system by running ‘herd status’ as root. > > IOW, Shepherd services are a special case of service. Just like D-Bus > services are another special case, etc. > > About logging: Shepherd does very little in that area. It limits itself > to providing a #:log-file parameter to capture the processes’s > stdout/stderr to a file. Other than that we usually configure daemons > to write to syslog, which provides more flexibility regarding storage > and filtering of log entries. Okay, this makes sense. I guess if I need filtering/searching I should probably look at packaging syslog-ng. IIRC all Shepherd services log to syslog, but do non-Shepherd services log somewhere? > > A “non-Shepherd service” as you call them doesn’t necessarily map to a > process, so there’s potentially nothing to log in the first place. It looks like Shepherd services don't necessarily map to a process either - on this graph: https://www.gnu.org/software/guix/manual/en/html_node/Shepherd-Services.html#Shepherd-Services various file systems, host-name, etc are also shown in the graph. > > Does that answer your questions? > > Thanks, > Ludo’. I think fundamentally what I'd most like to know is when should I use a Shepherd service vs a non-Shepherd service. Maybe it's as simple as: if you don't have any specific requirements always define a Shepherd service. It looks like both Shepherd services and non-Shepherd services: * Can have dependencies * Might not necessarily be a process * Can run an arbitrary expression which doesn't give me direction for why I should use one rather than the other. What might be confusing me is the documentation seems to imply a dichotomy of "Shepherd services" and "everything else", when perhaps there's actually a large number of service types in a rather flat hierarchy: * Shepherd services: start automatically, can be disabled, have separate start + stop methods * inet services: start on network request * mcron services: start at time * d-bus services: ?? * etc and Shepherd services are just one of many types of service differentiated by start conditions? This is speculation of course. In older distros I think perhaps the differentiation was more simple: services are processes started/stopped by run levels, and everything else was not a service. So services were like X11, inetd, and then there were non-service things like fstab, cron entries, inetd "services" configured in your inetd config file, etc. In systemd everything's a "unit" which can start/stop a process and form a graph, but then you have specialized hook units like "sockets" and "timers" that will control other units which are outside the graph (and thus won't be started based on graph order). The whole graph is executed when the system starts. As a metaphor, maybe Shepherd services would be similar to the plain units in systemd. Beyond what a service actually is though I have a few more questions: * Both https://www.gnu.org/software/guix/manual/en/html_node/Service-Composition.html and https://www.gnu.org/software/guix/manual/en/html_node/Shepherd-Services.html#Shepherd-Services appear to show a dependency graph. Are the dependency graphs Shepherd and non-Shepherd services entirely separate? Or maybe I'm completely misunderstanding "extension" in this context. Can an inet service depend on a non-inet service? Can an inet service depend on a d-bus service? * Is there a way to hook into service events - that is, run some code when a service starts or stops? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-04-18 15:11 ` rendaw @ 2019-04-19 12:09 ` Ludovic Courtès 2019-04-19 15:47 ` rendaw 0 siblings, 1 reply; 9+ messages in thread From: Ludovic Courtès @ 2019-04-19 12:09 UTC (permalink / raw) To: rendaw; +Cc: help-guix Hi, rendaw <7e9wc56emjakcm@s.rendaw.me> skribis: > I think fundamentally what I'd most like to know is when should I use a > Shepherd service vs a non-Shepherd service. Maybe it's as simple as: if > you don't have any specific requirements always define a Shepherd service. It’s hard to answer that question in the abstract. Do you have an example in mind that we could work through? > As a metaphor, maybe Shepherd services would be similar to the plain > units in systemd. Yes, I think so. > Beyond what a service actually is though I have a few more questions: > > * Both > https://www.gnu.org/software/guix/manual/en/html_node/Service-Composition.html > and > https://www.gnu.org/software/guix/manual/en/html_node/Shepherd-Services.html#Shepherd-Services > appear to show a dependency graph. The first page shows a service extension graph. The second page shows a graph of dependencies among Shepherd services. These are two different beasts. > Are the dependency graphs Shepherd and non-Shepherd services > entirely separate? Or maybe I'm completely misunderstanding > "extension" in this context. Can an inet service depend on a non-inet > service? Can an inet service depend on a d-bus service? * Is there a > way to hook into service events - that is, run some code when a > service starts or stops? An inetd service cannot “depend” on a non-inetd service; a D-Bus service cannot depend on a non-D-Bus service. Both D-Bus and inetd have their own notion of what a service is, how to start it, etc., which is separate from what the Shepherd does. I reckon that calling everything a “service” does not help understand all this… HTH, Ludo’. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-04-19 12:09 ` Ludovic Courtès @ 2019-04-19 15:47 ` rendaw 2019-05-04 7:01 ` Chris Marusich 0 siblings, 1 reply; 9+ messages in thread From: rendaw @ 2019-04-19 15:47 UTC (permalink / raw) To: Ludovic Courtès; +Cc: help-guix On 4/19/19 9:09 PM, Ludovic Courtès wrote: > Hi, > > rendaw <7e9wc56emjakcm@s.rendaw.me> skribis: > >> I think fundamentally what I'd most like to know is when should I use a >> Shepherd service vs a non-Shepherd service. Maybe it's as simple as: if >> you don't have any specific requirements always define a Shepherd service. > It’s hard to answer that question in the abstract. Do you have an > example in mind that we could work through? Yeah, so the basic example I was thinking of is I have an executable binary (a server for diagnostics) I'd like to run when the computer boots. > >> Beyond what a service actually is though I have a few more questions: >> >> * Both >> https://www.gnu.org/software/guix/manual/en/html_node/Service-Composition.html >> and >> https://www.gnu.org/software/guix/manual/en/html_node/Shepherd-Services.html#Shepherd-Services >> appear to show a dependency graph. > The first page shows a service extension graph. > > The second page shows a graph of dependencies among Shepherd services. > > These are two different beasts. Ah okay, that explains a lot - I was completely misinterpreting the extension graph then. I'm afraid I don't know what extension means in this context. Is it similar to OOP extension, where a "Dog" is-a "Canine"? Looking at the graphic, "accounts" depends-on "etc" (the directory/mount?) makes a lot more sense to me than "accounts" is-a "etc". Similarly I would assume that the udev service runs udevd, but if upower and colord are both "udev" services (because they extend udev) does that mean if I run both the upower and colord services then my system will have 3 instances of udevd running? I'm fairly sure that's not what it means, but if extends isn't an inheritance relationship and it's not a dependency relationship I'm not sure what it is. Is the udev service in the shepherd graph different then the udev service (-type?) in the service extension graph? That might be a factor in my confusion. >> Are the dependency graphs Shepherd and non-Shepherd services >> entirely separate? Or maybe I'm completely misunderstanding >> "extension" in this context. Can an inet service depend on a non-inet >> service? Can an inet service depend on a d-bus service? * Is there a >> way to hook into service events - that is, run some code when a >> service starts or stops? > An inetd service cannot “depend” on a non-inetd service; a D-Bus service > cannot depend on a non-D-Bus service. Both D-Bus and inetd have their > own notion of what a service is, how to start it, etc., which is > separate from what the Shepherd does. So suppose I have an mcron job that needs an ssh tunnel managed by shepherd to be started, or an inetd handler that needs data files on a network mount to run... is it possible to express those ordering dependencies? Or a service that subscribes the server to external webhooks, and needs the webhook listeners (inetd handlers) to be started before registering. I assume the answer to my 2nd question from the previous email - if there's a way to run code when a service's state changes - is that there currently isn't such hooking mechanism. > > I reckon that calling everything a “service” does not help understand > all this… I don't think it's an unreasonable choice of words :) - there's just a lot that isn't fitting in mentally for me. And thank you for your patience with the answers, I think I'm drawing much closer to understanding all this. Once I've figured it out I'll definitely write something up in case someone else has the same issues. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-04-19 15:47 ` rendaw @ 2019-05-04 7:01 ` Chris Marusich 2019-05-04 8:01 ` rendaw 0 siblings, 1 reply; 9+ messages in thread From: Chris Marusich @ 2019-05-04 7:01 UTC (permalink / raw) To: rendaw; +Cc: help-guix [-- Attachment #1: Type: text/plain, Size: 3079 bytes --] rendaw <7e9wc56emjakcm@s.rendaw.me> writes: > I'm fairly sure that's not what it means, but if extends isn't an > inheritance relationship and it's not a dependency relationship I'm not > sure what it is. In this context, when we say that a service A extends a service B, my understanding is that we mean that A changes (customizes) something about B. It is definitely not inheritance. And because A changes something about B, I don't think "A depends on B" is the best way to think about it. For example, the guix-service-type extends the account-service-type. The former provides the latter with a list of user and group accounts. In this way, the guix-service-type tells the account-service-type what users and groups the guix service needs in order to run. When you add a service of the guix-service-type to your operating system declaration's list of services (it's already there by default), the account-service-type will know what users and groups are required by the guix-service-type, and it will create them automatically. We say the guix-service-type "extends" the account-service-type because the former provides the latter with a list of users/groups to create. That is what we mean by "extension". Via this "extension" mechanism, it is possible to make cross-cutting changes that affect many different parts of the system, without making a bunch of changes to a bunch of services. For example, if the tor-service-type also needs to create a user account and group in order to do its job, it can simply extend the activation-service-type, just like the guix-service-type did. And if something about the way we create user accounts or groups needs to change, we can change it for all users and groups just by modifying one place: the account-service-type. You might be wondering: if the account-service-type is extended by user and group accounts, how are other services extended? The answer is that the type of object used depends on how the services are defined. The services participating in the extension define their own contract, within the confines of the general "extension mechanism" that Guix provides. Many services document how they can be extended in the manual, but some do not, and you might have to look at the source or existing examples to figure it out, which is not so great. Note that if you dig into the code, you will find that the term "extension" can also refer to a specific record type called <service-extension>, described in the "Service Reference" section of the manual, and defined in the gnu/services.scm file. If you want to learn more about it, I suggest reading the manual and also the source code - specifically the 'fold-services' procedure in gnu/services.scm, which is where the actual extension mechanism is implemented. The manual section is helpful, but due to some word ambiguities, it is a little difficult to follow the first time through. Some diagrams could help a lot in explaining the concepts. I hope this email helped a little. -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-05-04 7:01 ` Chris Marusich @ 2019-05-04 8:01 ` rendaw 2019-05-04 8:24 ` Chris Marusich 0 siblings, 1 reply; 9+ messages in thread From: rendaw @ 2019-05-04 8:01 UTC (permalink / raw) To: Chris Marusich; +Cc: help-guix On 5/4/19 4:01 PM, Chris Marusich wrote: > rendaw <7e9wc56emjakcm@s.rendaw.me> writes: > >> I'm fairly sure that's not what it means, but if extends isn't an >> inheritance relationship and it's not a dependency relationship I'm not >> sure what it is. > In this context, when we say that a service A extends a service B, my > understanding is that we mean that A changes (customizes) something > about B. It is definitely not inheritance. And because A changes > something about B, I don't think "A depends on B" is the best way to > think about it. > > For example, the guix-service-type extends the account-service-type. > The former provides the latter with a list of user and group accounts. > In this way, the guix-service-type tells the account-service-type what > users and groups the guix service needs in order to run. When you add a > service of the guix-service-type to your operating system declaration's > list of services (it's already there by default), the > account-service-type will know what users and groups are required by the > guix-service-type, and it will create them automatically. We say the > guix-service-type "extends" the account-service-type because the former > provides the latter with a list of users/groups to create. > > That is what we mean by "extension". Via this "extension" mechanism, it > is possible to make cross-cutting changes that affect many different > parts of the system, without making a bunch of changes to a bunch of > services. For example, if the tor-service-type also needs to create a > user account and group in order to do its job, it can simply extend the > activation-service-type, just like the guix-service-type did. And if > something about the way we create user accounts or groups needs to > change, we can change it for all users and groups just by modifying one > place: the account-service-type. > > You might be wondering: if the account-service-type is extended by user > and group accounts, how are other services extended? The answer is that > the type of object used depends on how the services are defined. The > services participating in the extension define their own contract, > within the confines of the general "extension mechanism" that Guix > provides. Many services document how they can be extended in the > manual, but some do not, and you might have to look at the source or > existing examples to figure it out, which is not so great. > > Note that if you dig into the code, you will find that the term > "extension" can also refer to a specific record type called > <service-extension>, described in the "Service Reference" section of the > manual, and defined in the gnu/services.scm file. If you want to learn > more about it, I suggest reading the manual and also the source code - > specifically the 'fold-services' procedure in gnu/services.scm, which is > where the actual extension mechanism is implemented. Wow, thanks for taking the time to write this up! I believe that's a much clearer explanation - I actually figured it out by reading the source code extensively, but I thought this thread had been abandoned. I've written up something similar here: https://gitlab.com/rendaw/blog/blob/master/how_to_guix_for_those_who_dont.md I think it would be great to have this information in the documentation! I'd also elaborate on what actually happens in the end ("If every service just extends other services, what does the root service do?"), how stuff actually runs (since operating systems are processes and not data after all -> I think the answer is that the boot service, activation service, and shepherd service all run processes but that leads to a lot of other obvious questions: what's the difference between the boot and activation services - or what do each do and what makes them suitable for those tasks, what order are processes started in each of them, how does guix deal with existing files created by activation processes when reconfiguring the system, etc), what is a service vs a service type, and an explanation of the folding process and how it relates to a service type definition. > The manual section is helpful I strongly believe the manual section is not helpful to anyone who doesn't already know how Guix works. The explanation of _everything_ related to services hinges on the word "extend" - and it's never defined! It's a novel definition in this context, it's not a matter of ambiguity. The definition of extends here is strongly tied to the folding procedure so both of those need to be in the documentation for a reader to gather an understanding of what's being discussed. Reading the source code is a colossal hurdle for users, and especially considering that this is a core service and the actual behavior is influenced by and influences hundreds of other pieces of code (unlike things like `(shell ...)` which get passed through to the end system interacting with relatively few other processes). Knowing the big picture of how Guix works as a system, how components interact under the hood, etc. is critical because it lets you make an informed decision when you want to create a new service, it gives you hints on how to read arcane build errors, when the resulting system doesn't work it lets you know where things could be going wrong, and if you want to read more about a particular subject you know what topics/search terms/articles could be relevant. I hope this doesn't come off too aggressive. I view myself as a user at this point and really want Guix to take off. I spent the greater part of three weeks trying how to figure out how to get my new (fairly simple) Guix system running. IMO if it takes most users (who want to do more than install X and then run ./x) three full-time weeks, breathing Guile, and source code diving to reach a usable point then Guix will never be able to achieve critical mass. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-05-04 8:01 ` rendaw @ 2019-05-04 8:24 ` Chris Marusich 2019-05-04 8:32 ` rendaw 0 siblings, 1 reply; 9+ messages in thread From: Chris Marusich @ 2019-05-04 8:24 UTC (permalink / raw) To: rendaw; +Cc: help-guix [-- Attachment #1: Type: text/plain, Size: 2841 bytes --] rendaw <7e9wc56emjakcm@s.rendaw.me> writes: > I've written up something similar here: > https://gitlab.com/rendaw/blog/blob/master/how_to_guix_for_those_who_dont.md Nice! Thank you for sharing it. > I think it would be great to have this information in the > documentation! I agree, but I am not sure how to merge the important parts with the documentation in a way that does not clutter it up and make it even more confusing than it already is. > I'd also elaborate on what actually happens in the end > ("If every service just extends other services, what does the root > service do?"), how stuff actually runs (since operating systems are > processes and not data after all -> I think the answer is that the boot > service, activation service, and shepherd service all run processes but > that leads to a lot of other obvious questions: what's the difference > between the boot and activation services - or what do each do and what > makes them suitable for those tasks, what order are processes started in > each of them, how does guix deal with existing files created by > activation processes when reconfiguring the system, etc), what is a > service vs a service type, and an explanation of the folding process and > how it relates to a service type definition. Yes, explaining this could make for a good manual section or blog post. >> The manual section is helpful > > I strongly believe the manual section is not helpful to anyone who > doesn't already know how Guix works. It's true that we could do a better job of introducing services in the manual. > Reading the source code is a colossal hurdle for users I think this statement is perhaps over-broad, since "reading the source" can mean "reading an operating system configuration file," which is definitely easier than understanding Guix's services. That said, I agree that users shouldn't have to grok fold-services just to understand how to use services in their operating system configuration. The manual tries to explain services, but it just doesn't get the job done, and we should improve it. > I hope this doesn't come off too aggressive. I don't think it does, and I appreciate hearing your opinion. I actually felt exactly the same way as you, when I was first learning about services. It was frustrating. As I learned, I submitted patches to change the documentation where it was unclear or incorrect. But it still isn't good enough. I've been thinking about trying to rewrite that section in a way that makes it easier to understand services for a first-time reader, but I haven't yet done it. If you have ideas about how to improve the documentation, please consider submitting a patch. Coding can be hard, but so can documentation. Every little improvement can help! -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Services and log management/monitoring 2019-05-04 8:24 ` Chris Marusich @ 2019-05-04 8:32 ` rendaw 0 siblings, 0 replies; 9+ messages in thread From: rendaw @ 2019-05-04 8:32 UTC (permalink / raw) To: Chris Marusich; +Cc: help-guix On 5/4/19 5:24 PM, Chris Marusich wrote: > I've been thinking about trying to rewrite > that section in a way that makes it easier to understand services for a > first-time reader, but I haven't yet done it. > Ah, I'm totally for that! I thought though if my previous post wasn't over the line, suggesting (or contributing) a complete rewrite would. I'd offer the section from my guide on services but as I mentioned before I hand waved some stuff that I never managed to figure out (activation/boot service task ordering, how they handle failures, etc). ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-05-04 8:32 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-04-14 18:12 Services and log management/monitoring rendaw 2019-04-17 21:20 ` Ludovic Courtès 2019-04-18 15:11 ` rendaw 2019-04-19 12:09 ` Ludovic Courtès 2019-04-19 15:47 ` rendaw 2019-05-04 7:01 ` Chris Marusich 2019-05-04 8:01 ` rendaw 2019-05-04 8:24 ` Chris Marusich 2019-05-04 8:32 ` rendaw
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).