From: "Ludovic Courtès" <ludo@gnu.org>
To: guix-devel@gnu.org
Subject: The Shepherd on Fibers
Date: Wed, 23 Mar 2022 23:36:30 +0100 [thread overview]
Message-ID: <87ee2sfg9d.fsf@inria.fr> (raw)
Hello Guix!
I have pushed a ‘wip-fibers’ branch of the Shepherd:
https://git.savannah.gnu.org/cgit/shepherd.git/log/?h=wip-fibers
The goal is to make shepherd (the daemon) use Fibers¹ for concurrency.
Right now, actions in shepherd are all serialized: starting services,
waiting for their PID files, accepting client connections and serving
them (when running the ‘herd’ command), etc. This slows down startup,
prevents things like running two clients at the same time, or serving a
client while waiting for a PID file. In other words, it sucks.
Using Fibers, we can introduce concurrency with few changes to the code;
we can even do new things writing code that looks sequential. This is
what the branch does.
To illustrate that, it introduces a new, incredible feature: logging!
As you know, shepherd had (euphemism ahead) limited support for logging,
essentially in the form of #:log-file, which would redirect a daemon’s
stdout/stderr to a file, and also in the form of “let’s hope the daemon
talks to syslogd”. Here I was able to trivially add logging, such that
anything spawned by ‘fork+exec-command’ is logged, with timestamps and
all (the logger is a fiber that calls ‘read-line’ in a loop, which is
automagically non-blocking—delightful!). The next fun step (besides
logging) will be adding inetd-style and/or systemd-style “socket
activation”.
Fibers is used in a single-threaded fashion, which is the main
constraint for shepherd since it forks. That also means that fibers
cannot be preempted, so it’s fully cooperative scheduling.
There’s one catch: Fibers is currently Linux-only. The good news is
that work has been done to port it to other kernels via libevent².
Until it is merged, we could keep using the Shepherd 0.8 on GNU/Hurd.
I’ve done some Guix System testing in VMs and didn’t notice any major
issues. I’d like to merge that branch in ‘master’ and to eventually
release it as 0.9.0 (with or without socket activation, we’ll see.)
Hopefully, we could be running it within a couple of weeks.
Thoughts?
Ludo’.
¹ https://github.com/wingo/fibers
² https://github.com/wingo/fibers/pull/53
next reply other threads:[~2022-03-23 22:37 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-03-23 22:36 Ludovic Courtès [this message]
2022-03-25 13:29 ` The Shepherd on Fibers Maxim Cournoyer
2022-03-26 21:28 ` Ludovic Courtès
2022-03-26 11:06 ` pelzflorian (Florian Pelz)
2022-03-26 11:09 ` Zhu Zihao
2022-03-26 11:16 ` Zhu Zihao
2022-03-26 11:18 ` pelzflorian (Florian Pelz)
2022-03-26 11:27 ` Zhu Zihao
2022-03-26 16:56 ` pelzflorian (Florian Pelz)
2022-03-26 11:59 ` Maxime Devos
2022-03-26 16:52 ` pelzflorian (Florian Pelz)
2022-03-26 18:20 ` Maxime Devos
2022-03-29 11:44 ` Ludovic Courtès
2022-03-26 12:03 ` Maxime Devos
2022-03-29 12:47 ` Ludovic Courtès
2022-03-26 12:11 ` Maxime Devos
2022-03-29 12:48 ` Ludovic Courtès
2022-03-29 16:26 ` Maxime Devos
2022-03-30 15:14 ` Ludovic Courtès
2022-03-30 17:16 ` Maxime Devos
2022-03-26 12:16 ` Maxime Devos
2022-03-29 12:50 ` Ludovic Courtès
2022-03-29 12:52 ` Maxime Devos
2022-03-29 12:54 ` Maxime Devos
2022-03-29 15:29 ` Attila Lendvai
2022-03-30 10:05 ` Ludovic Courtès
2022-03-31 4:33 ` adriano
2022-03-31 7:56 ` Attila Lendvai
2022-03-26 12:44 ` Maxime Devos
2022-03-29 13:03 ` Ludovic Courtès
2022-03-26 19:24 ` Maxime Devos
2022-03-29 0:14 ` Philip McGrath
2022-03-29 0:22 ` Philip McGrath
2022-03-29 9:36 ` Maxime Devos
2022-03-29 11:11 ` Philip McGrath
2022-03-30 10:00 ` Ludovic Courtès
2022-03-29 10:13 ` Zhu Zihao
2022-03-29 10:40 ` Maxime Devos
2022-03-30 9:49 ` Ludovic Courtès
2022-03-29 13:16 ` Ludovic Courtès
-- strict thread matches above, loose matches on Subject: below --
2022-03-24 6:48 Brendan Tildesley
2022-03-24 16:57 Nathan Dehnel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ee2sfg9d.fsf@inria.fr \
--to=ludo@gnu.org \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).