bug#22814: 25.0.91; Emacs runs out of file descriptors on OS X

[Anders Lindgren <andlind@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2524 bytes --]

Hi!

I just gave this some thought (without knowing anything about the kqueue
notifications).

I don't think the problem is what kqueue does when it run out of file
descriptors, but how the rest of the Emacs process acts when this happens.
For example, can it even read and write files? Can subprocesses be started?

The question, then, is how should the kqueue system work so that it doesn't
run Emacs out of file descriptors? For example, if it would be possible to
check how many are available, it could stop when there are, say, 50
remaining. Effectively, this would give a user 200 files that are
auto-reverted using the notification system, and the rest would be handled
by the old timer system.

    -- Anders

On Sat, Feb 27, 2016 at 2:26 PM, Michael Albinus <michael.albinus@gmx.de>
wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> >> > One simple way to handle this is to define a variable with "max"
> >> > number of files the notification system can handle. We can set this
> >> > to, say, 200 on OS X and unlimited on other systems.
> >>
> >> Would be possible, yes. I would prefer to set the limit to a system
> >> related value. Does there exist a portable way to detect, how many file
> >> descriptors / processes Emacs is able to consume?
> >
> > Not portably, AFAIK.
>
> On POSIX systems, we could propagate the result of
>
> if !(getrlimit (RLIMIT_NOFILE, &rlim))
>    return make_number (rlim.rlim_cur);
> else
>    return Qnil;
>
> Maybe we add a function like `<library>-max-descriptors' to the
> libraries. Or maybe not, and the backends check this themselves, and
> cease to work when reaching an internal limit. Such an internal limit
> could be half the number of soft RLIMIT_NOFILE on POSIX systems, for
> example.
>
> > Also, different implementations use different
> > resources for receiving file notifications.  For example, w32notify
> > uses one handle and one thread per watched file/directory.  The number
> > of handles a process can have on Windows is very large, and the
> > theoretical max number of threads is 32K, but both are limited by the
> > resources already consumed by the Emacs process.  So determining the
> > practical maximum at any given point will require a non-trivial
> > function.
>
> There must be different scenarios for different file notification
> libraries. But using RLIMIT_NOFILE as basis when possible gives us
> better results on systems like OS X and FreeBSD, which both use kqueue,
> but provide different ressource limits.
>
> Best regards, Michael.
>

[-- Attachment #2: Type: text/html, Size: 3248 bytes --]