Using CHILD_SUBREAPER in GNU Shepherd

Using CHILD_SUBREAPER in GNU Shepherd

[sbaugh]

Hi,

There is a feature present in Linux: CHILD_SUBREAPER.

It changes the logic for reparenting orphaned processes. Instead of an
orphaned process being reparented to pid1, an orphaned process is
reparented to the nearest parent that is marked as a CHILD_SUBREAPER.
A process can mark itself as a CHILD_SUBREAPER without privileges.

See PR_SET_CHILD_SUBREAPER in prctl(2) for maybe more precise
information: http://man7.org/linux/man-pages/man2/prctl.2.html

What this boils down to is allowing process supervisors to be much more
reliable, even when not running as init, because they can track not just
their children, but their children's children, and in general all
transitive children. I'd like to add it to GNU Shepherd.

It may require some re-architecting to take full advantage of it. I'm
not sure yet.  I wrote a small tool using CHILD_SUBREAPER to provide
some useful process supervision features:
https://github.com/catern/supervise

The relevant features are:
- Guaranteed cleanup of all started processes
- Usable in nested situations
I'd like to get such features into GNU Shepherd. (maybe the latter is
already possible, but explicit support can't hurt)

Does this sound like a good idea?

There aren't many process supervisors out there which actually use
CHILD_SUBREAPER, which I find rather disappointing, because it allows
container-like cleanup without actually having root privileges.

In the longer term, if we had this feature in the shepherd, we could
make some container-like guarantees about starting up daemons and
applications out of Guix on foreign distros: Not only will the
dependencies be pulled from the store, but also absolutely no processes
can be remaining on the system after the daemon is terminated, if it is
started with the shepherd. That would a really unique guarantee!

It could also help with store garbage collection, perhaps?

Thanks for Guix!

Re: Using CHILD_SUBREAPER in GNU Shepherd

[Ludovic Courtès]

Hello!

sbaugh@catern.com skribis:

> What this boils down to is allowing process supervisors to be much more
> reliable, even when not running as init, because they can track not just
> their children, but their children's children, and in general all
> transitive children. I'd like to add it to GNU Shepherd.
>
> It may require some re-architecting to take full advantage of it. I'm
> not sure yet.  I wrote a small tool using CHILD_SUBREAPER to provide
> some useful process supervision features:
> https://github.com/catern/supervise
>
> The relevant features are:
> - Guaranteed cleanup of all started processes
> - Usable in nested situations
> I'd like to get such features into GNU Shepherd. (maybe the latter is
> already possible, but explicit support can't hurt)
>
> Does this sound like a good idea?

It does!  We want to keep the Shepherd portable to GNU variants that
lack CHILD_SUBREAPER, so we’ll have to pay attention to it, but
hopefully that won’t be much of a problem.

There are prctl bindings in (guix build syscalls) so perhaps you could
copy/paste that (yeah…) in the Shepherd to begin with.

> There aren't many process supervisors out there which actually use
> CHILD_SUBREAPER, which I find rather disappointing, because it allows
> container-like cleanup without actually having root privileges.
>
> In the longer term, if we had this feature in the shepherd, we could
> make some container-like guarantees about starting up daemons and
> applications out of Guix on foreign distros: Not only will the
> dependencies be pulled from the store, but also absolutely no processes
> can be remaining on the system after the daemon is terminated, if it is
> started with the shepherd. That would a really unique guarantee!
>
> It could also help with store garbage collection, perhaps?

Hmm I don’t think it would help with GC of /gnu/store items.  Or did you
mean something else?

Thanks for the suggestion, looking forward to a WIP patch!  :-)

Ludo’.