all messages for Guix-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
* setting open files limit for daemon processes
@ 2022-02-25  7:55 Attila Lendvai
  2022-02-25 10:42 ` Maxime Devos
  0 siblings, 1 reply; 5+ messages in thread
From: Attila Lendvai @ 2022-02-25  7:55 UTC (permalink / raw)
  To: guix-devel

dear Guix'ers,

"The per-process limit is inherited by each process from its parent", and Shepherd is the init process. when it spawns a daemon, it inherits its open files limit.

i have successfully set the limit for the daemon user using:

(pam-limits-service
  (list
    (pam-limits-entry "*" 'both 'nofile 100000)))

and it is applied as observable with:

su - [daemon user] -c 'ulimit -aHS' -s `which bash`

and yet, when i `herd stop` and `herd start` the process, even though it's running with the right uid, its limit is still the default, as per:

cat /proc/[pid]/limits

the suggested solution i find online is to edit:

/etc/pam.d/common-session-noninteractive

to include:

session required pam_limits.so

but this is not how Guix arranges things.

i looked into extending my service (https://github.com/attila-lendvai/guix-crypto/blob/main/src/guix-crypto/services/swarm.scm) to add some pam rules, but i'd need to learn a whole lot of pam for that, and the Guix code around service extensions is also rather complex, so i decided to ask here:

is there an example somewhere demonstrating how to increase the open file limit of a daemon process?

any hints/guidance is appreciated.

--
- attila
PGP: 5D5F 45C7 DFCD 0A39n


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: setting open files limit for daemon processes
  2022-02-25  7:55 setting open files limit for daemon processes Attila Lendvai
@ 2022-02-25 10:42 ` Maxime Devos
  2022-02-25 14:09   ` Attila Lendvai
  0 siblings, 1 reply; 5+ messages in thread
From: Maxime Devos @ 2022-02-25 10:42 UTC (permalink / raw)
  To: Attila Lendvai, guix-devel

[-- Attachment #1: Type: text/plain, Size: 954 bytes --]

Attila Lendvai schreef op vr 25-02-2022 om 07:55 [+0000]:
> "The per-process limit is inherited by each process from its parent",
> and Shepherd is the init process. when it spawns a daemon, it inherits
>  its open files limit.
> 
> i have successfully set the limit for the daemon user using:
> 
> (pam-limits-service
>   (list
>     (pam-limits-entry "*" 'both 'nofile 100000)))
> 
> and it is applied as observable with:
> 
> su - [daemon user] -c 'ulimit -aHS' -s `which bash`

That might set the limit of the user when that user logins (and hence,
PAM things are run), but I don't see how this changes the limit of
shepherd itself.  I don't think that shepherd interacts with PAM at
all?

My suggestion is to do (setrlimit RLIMIT_NOFILE [...]) inside shepherd
itself -- when shepherd starts, or between 'fork' and 'exec'.  Maybe
an '#:open-file-limit' argument could be added to 'fork+exec-command'?

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: setting open files limit for daemon processes
  2022-02-25 10:42 ` Maxime Devos
@ 2022-02-25 14:09   ` Attila Lendvai
  2022-02-25 21:28     ` Attila Lendvai
  0 siblings, 1 reply; 5+ messages in thread
From: Attila Lendvai @ 2022-02-25 14:09 UTC (permalink / raw)
  To: Maxime Devos; +Cc: guix-devel

> > su - [daemon user] -c 'ulimit -aHS' -s `which bash`
>
> That might set the limit of the user when that user logins (and hence,
> PAM things are run), but I don't see how this changes the limit of
> shepherd itself. I don't think that shepherd interacts with PAM at
> all?


my understanding of PAM is rather limited, but i guess it cannot hook into
setuid(), and as such it has no means to affect the ulimits of processes spawned
by Shepherd.


> My suggestion is to do (setrlimit RLIMIT_NOFILE [...]) inside shepherd
> itself -- when shepherd starts, or between 'fork' and 'exec'. Maybe


looking at the code, it'd be nice if we factored out a variant of fork (maybe
called CALL-IN-FORK) that took a thunk and called it in the forked path. that
would allow me to use that abstraction in user code to easily insert a call to
setrlimit before the EXEC-COMMAND, or whatever else is needed.

maybe using that abstraction we could straight out move EXEC-COMMAND to the guix
side? my thinking here is that toching/testing/updating the Shepherd codebase
seems to be much more trouble than the Guix codebase.

i'd be happy to play with this, but i don't know how to run a Guix VM that is
built using my modified Shepherd; i.e. i have no idea how to test what i'm
doing.


> an '#:open-file-limit' argument could be added to 'fork+exec-command'?


that would be the safest/simplest way to resolve this, but then what about all
the other limits?

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“There can be no keener revelation of a society’s soul than the way in which it treats its children.”
	— Nelson Mandela (1918–2013)



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: setting open files limit for daemon processes
  2022-02-25 14:09   ` Attila Lendvai
@ 2022-02-25 21:28     ` Attila Lendvai
  2022-02-25 22:04       ` Maxime Devos
  0 siblings, 1 reply; 5+ messages in thread
From: Attila Lendvai @ 2022-02-25 21:28 UTC (permalink / raw)
  To: Attila Lendvai; +Cc: guix-devel

> i'd be happy to play with this, but i don't know how to run a Guix VM that is
> built using my modified Shepherd; i.e. i have no idea how to test what i'm
> doing.

it actually seems simpler than i thought: i edited the Shepherd package to:

(source (git-checkout (url "file://my-path/shepherd/")))

plus i added some extra NATIVE-INPUTS to be able to build it from the unreleased
sources. after that a `./pre-inst-env guix system vm ...` seems to pick up the
changes, this would be a bearable edit-build-test cycle.

except that modifying shepherd seems to trigger quite a lot of packages to be
rebuilt.

is there a way to instruct `guix system vm ...` _not_ to rebuild all the
dependants, and only replace the Shepherd that is running as the init process?

is this something where grafts could be helpful?

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“Awareness isn’t something we own; awareness isn’t something we possess. Awareness is actually what we are.”
	— Adyashanti (1962–)



^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: setting open files limit for daemon processes
  2022-02-25 21:28     ` Attila Lendvai
@ 2022-02-25 22:04       ` Maxime Devos
  0 siblings, 0 replies; 5+ messages in thread
From: Maxime Devos @ 2022-02-25 22:04 UTC (permalink / raw)
  To: Attila Lendvai; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]

Attila Lendvai schreef op vr 25-02-2022 om 21:28 [+0000]:
> is there a way to instruct `guix system vm ...` _not_ to rebuild all the
> dependants, and only replace the Shepherd that is running as the init process?

This can be done by customising the 'shepherd' field of 'shepherd-
configuration', introduced in 95f72dcd7aece05e9252c93bef5a831f96cb5393.

The manual has an example on how to use this:

The following example specifies the Shepherd package for the operating
system:

@lisp
(operating-system
  ;; ...
  (services (append (list openssh-service-type))
            ;; ...
            %desktop-services)
  ;; ...
  ;; Use own Shepherd package.
  (essential-services
   (modify-services (operating-system-default-essential-services
                     this-operating-system)
     (shepherd-root-service-type config => (shepherd-configuration
                                            (inherit config)
                                            (shepherd my-shepherd))))))

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2022-02-25 23:33 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-02-25  7:55 setting open files limit for daemon processes Attila Lendvai
2022-02-25 10:42 ` Maxime Devos
2022-02-25 14:09   ` Attila Lendvai
2022-02-25 21:28     ` Attila Lendvai
2022-02-25 22:04       ` Maxime Devos

Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/guix.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.