bug#39164: 27.0.60; Intermittent crash on MacOS 10.14 in setup_process_coding_systems

[YAMAMOTO Mitsuharu <mituharu@math.s.chiba-u.ac.jp>]

On Tue, 28 Jan 2020 16:59:35 +0900,
Robert Pluim wrote:
> 
> >>>>> On Mon, 27 Jan 2020 20:19:07 +0200, Eli Zaretskii <eliz@gnu.org> said:
> 
>     >> Date: Mon, 27 Jan 2020 22:26:10 +0900
>     >> From: mituharu@math.s.chiba-u.ac.jp
>     >> Cc: "Eli Zaretskii" <eliz@gnu.org>,
>     >> jguenther@gmail.com,
>     >> "Tomasz Kowal" <tomekowal@gmail.com>,
>     >> 39164@debbugs.gnu.org
>     >> 
>     >> This is a variant of Bug#24325: Crash - fd larger than FD_SETSIZE.
>     >> Some functions in the Core Foundation framework on macOS
>     >> call setrlimit for RLIMIT_NOFILE in order to increase the limit of
>     >> the maximum number of open files for the process:
>     >> 
>     >> https://opensource.apple.com/source/CF/CF-1153.18/CFSocket.c.auto.html
>     >> 
>     >> So the fix for Bug#24325 doesn't work in this case.
> 
>     Eli> Thanks.
> 
>     Eli> So I guess one possible solution would be to see that fd is beyond
>     Eli> FD_SETSIZE, and if so, call getrlimit to see if the limit was bumped
>     Eli> up, and if so, reallocate the arrays used by process.c?  Would that
>     Eli> make sense?
> 
> This will break {p}select, no? Thatʼs defined to only work up to
> FD_SETSIZE.

Yes.  I'm currently thinking about adding the following workaround to
the next release of the Mac port.  It is not a perfect solution, but
the Mac port is still based on Emacs 26, and I don't want to make a
drastic change there.

			     YAMAMOTO Mitsuharu
			mituharu@math.s.chiba-u.ac.jp

diff --git a/src/macappkit.m b/src/macappkit.m
index d2647aff97..08a7e6bc9b 100644
--- a/src/macappkit.m
+++ b/src/macappkit.m
@@ -1337,6 +1337,21 @@ - (void)applicationDidFinishLaunching:(NSNotification *)notification
   setenv ("CAVIEW_USE_GL", "1", 0);
 #endif
 
+  /* Some functions/methods in CoreFoundation/Foundation increase the
+     maximum number of open files for the process in their first call.
+     We make dummy calls to them and then reduce the resource limit
+     here, since pselect cannot handle file descriptors that are
+     greater than or equal to FD_SETSIZE.  */
+  CFSocketGetTypeID ();
+  CFFileDescriptorGetTypeID ();
+  MRC_RELEASE ([[NSFileHandle alloc] init]);
+  struct rlimit rlim;
+  if (getrlimit (RLIMIT_NOFILE, &rlim) == 0 && rlim.rlim_cur > FD_SETSIZE)
+    {
+      rlim.rlim_cur = FD_SETSIZE;
+      setrlimit (RLIMIT_NOFILE, &rlim);
+    }
+
   /* Exit from the main event loop.  */
   [NSApp stop:nil];
   [NSApp postDummyEvent];