From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Robert Pluim Newsgroups: gmane.emacs.bugs Subject: bug#40023: 26.3 Emacs crashes when creating process if many file handles are in use (e.g., when using ccls) Date: Tue, 14 Apr 2020 15:58:51 +0200 Message-ID: References: <865E1017-DFA9-495D-BCDA-70B46FE09A27@pirk.name> <87pncdgg5b.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="77767"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= , 40023@debbugs.gnu.org To: Michael Albinus Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Apr 14 18:43:21 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jOOed-000K6c-OF for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 14 Apr 2020 18:43:19 +0200 Original-Received: from localhost ([::1]:34634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOOec-00025L-NS for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 14 Apr 2020 12:43:18 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57964) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOOeE-0001yz-SZ for bug-gnu-emacs@gnu.org; Tue, 14 Apr 2020 12:42:56 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jOOeD-0006ct-BE for bug-gnu-emacs@gnu.org; Tue, 14 Apr 2020 12:42:54 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:51201) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jOM6c-0007ma-Ig for bug-gnu-emacs@gnu.org; Tue, 14 Apr 2020 10:00:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jOM6c-0001nv-GL for bug-gnu-emacs@gnu.org; Tue, 14 Apr 2020 10:00:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Robert Pluim Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 14 Apr 2020 14:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 40023 X-GNU-PR-Package: emacs Original-Received: via spool by 40023-submit@debbugs.gnu.org id=B40023.15868727426830 (code B ref 40023); Tue, 14 Apr 2020 14:00:02 +0000 Original-Received: (at 40023) by debbugs.gnu.org; 14 Apr 2020 13:59:02 +0000 Original-Received: from localhost ([127.0.0.1]:34514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOM5e-0001m6-5V for submit@debbugs.gnu.org; Tue, 14 Apr 2020 09:59:02 -0400 Original-Received: from mail-wr1-f65.google.com ([209.85.221.65]:35095) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOM5c-0001lb-8s for 40023@debbugs.gnu.org; Tue, 14 Apr 2020 09:59:00 -0400 Original-Received: by mail-wr1-f65.google.com with SMTP id x18so10154134wrq.2 for <40023@debbugs.gnu.org>; Tue, 14 Apr 2020 06:59:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :mime-version:content-transfer-encoding; bh=b0LQlAC8ipYB9+HjHETqJxfm7W6YYatfxaO//mDL5vM=; b=G8DifTpycnm6bUmALv2AngtHJhYh7oIfSa9twh8c0P4tYVdPip2y/h0ETA/UVV8oC5 GRa+dcfG3UW0oQT2P9xVzfpJE9S2WLEH9+hByani/NS8jSvTV1oEPO4l+ThPSUdquxlx oHFVWh7Px8QcUzVLYZuZbSmgmk/+kFmpvt1Mn673MXtROtf+ZlRhPTajEC5CSU88RLOD PphaaBtYpX68gbcGJKQCElA9HkuX9qKyPWtGAqJrhqV2Ql8D+fEhRZDSPeGjVi3hISfS ulATAMfREoryM3qNpja7xOWmhFFVeX+pKGM4g0K41yvbZX95KRN54sxmzenZZDtaq8Bg kzrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:mime-version:content-transfer-encoding; bh=b0LQlAC8ipYB9+HjHETqJxfm7W6YYatfxaO//mDL5vM=; b=QeNilM978mnjQpfB7Oy+WsSwiwxT44j934qjR7SJmC1tm0t4Z5qganXat9Bko29FH6 LXSPt9XoxU7vq9pyAtaLEa0ajadUFG+jF+0qOv0C5yHaxcmeQXSlJ59/GI7zPSv/F4Yp RPr6ZfZRdQDH4S3/Gqsz3Gsoq0wfOoKDodRPQazf3OhpcUhg8JmIRseDlHJueA/Q3glP nTXKlBmIkpUJNyVKiul0ED146vyVwrUoAcwMshsEgELdg3t3uq+9LQbY3ssbAJuKQDzt quANZDIwdDF+24+3QzrlPAjbP8GAfO9IIPNIuOCZpxT5kAwsqIa7Jtxd7iPifGUVtp5Q vZVw== X-Gm-Message-State: AGi0PubiihkEjEx12v5bH83at/thOMO+7sHe12PRe51UvkKHhPInyeZj 1AlD0HiguB7COaKDOxZd8++Zg24M X-Google-Smtp-Source: APiQypJyh408fjni38/+2VK1XbJsx9QkySQFpPr4iZp/4/BeCvPjA4M43nHkGv3xY2wlVVxa6gU5AA== X-Received: by 2002:adf:e84e:: with SMTP id d14mr22420924wrn.67.1586872733868; Tue, 14 Apr 2020 06:58:53 -0700 (PDT) Original-Received: from rpluim-mac ([2a01:e34:ecfc:a860:f1a6:22e0:cfda:c9f2]) by smtp.gmail.com with ESMTPSA id c1sm18579166wrc.4.2020.04.14.06.58.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Apr 2020 06:58:53 -0700 (PDT) In-Reply-To: <87pncdgg5b.fsf@gmx.de> (Michael Albinus's message of "Sun, 12 Apr 2020 12:19:44 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:178363 Archived-At: >>>>> On Sun, 12 Apr 2020 12:19:44 +0200, Michael Albinus said: Michael> Mattias Engdeg=C3=A5rd writes: >> Actually reducing RLIMIT_NOFILE doesn't sound like a terrific idea -- >> if anything, we need as many descriptors as we can get, in particular >> on macOS where kqueue uses up one per monitored file. If we are >> limited by FD_SETSIZE, then we shouldn't use select. Or am I missing >> something? With lsp-mode I think it=CA=BCs one fd per directory containing a monitored file, but either way it=CA=BCs a limitation, and one that people are running into. Michael> kqueue limits itself to (RLIMIT_NOFILE - 50) file descriptors.= If 50 Michael> file descriptors aren't sufficient, we could increase that num= ber, or Michael> make it a defvar changeable via Lisp. 50 is enough. The issue is that ccls uses up the other FD_SETSIZE - 50 descriptors, and then other libraries end up passing descriptors > FD_SETSIZE back to emacs because RLIMIT_NOFILE has been changed behind our back. I only see two solutions: - the patch from , which reduces RLIMIT_NOFILE to FD_SETSIZE. It works for me, but I have to artificially generate the crashing conditions, so some feedback from others would be nice (especially if we want to put it in emacs-27). Plus it limits the number of open file descriptors even on systems where crazy users want to monitor 10k files (maybe that=CA=BCs a good thing :-) ). - select() -> poll(). GNU/Linux, macOS and *BSD all have poll, plus there=CA=BCs a gnulib module for it that we could perhaps use on MS-Windows. As Eli says, not a small change. (increasing FD_SETSIZE is a no-go: it works on macOS, maybe on *BSD, and is on very shaky standards-compliance ground). Robert