From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric?= Giquel via "Bug reports for GNU Emacs, the Swiss army knife of text editors" Newsgroups: gmane.emacs.bugs Subject: bug#55737: [PATCH] 28.1; Values of `read-process-output-max' above 64k not taken into account on GNU/Linux Date: Tue, 31 May 2022 13:09:38 +0200 Message-ID: <4088c6d9badbb3e2015277c8dfdb7541b8a5fd9f.camel@laposte.net> Reply-To: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric?= Giquel Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-LE+vfNM1KAgPBsk8Pp4K" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="11205"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Evolution 3.44.1-2 To: 55737@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue May 31 13:37:04 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nw0BM-0002l2-Ox for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 31 May 2022 13:37:04 +0200 Original-Received: from localhost ([::1]:49360 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nw0BL-0001g8-1V for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 31 May 2022 07:37:03 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51308) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzlC-0006H2-Ri for bug-gnu-emacs@gnu.org; Tue, 31 May 2022 07:10:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:52899) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nvzlC-0000Sm-Hx for bug-gnu-emacs@gnu.org; Tue, 31 May 2022 07:10:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nvzlC-0001lV-B4 for bug-gnu-emacs@gnu.org; Tue, 31 May 2022 07:10:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric?= Giquel Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 31 May 2022 11:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 55737 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: patch X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.16539953996769 (code B ref -1); Tue, 31 May 2022 11:10:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 31 May 2022 11:09:59 +0000 Original-Received: from localhost ([127.0.0.1]:46796 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nvzl9-0001l7-78 for submit@debbugs.gnu.org; Tue, 31 May 2022 07:09:59 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:57294) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nvzl6-0001ky-BE for submit@debbugs.gnu.org; Tue, 31 May 2022 07:09:57 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzl6-00065I-6S for bug-gnu-emacs@gnu.org; Tue, 31 May 2022 07:09:56 -0400 Original-Received: from smtp-outgoing-1801.laposte.net ([160.92.124.102]:50045) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nvzl3-0000I2-29 for bug-gnu-emacs@gnu.org; Tue, 31 May 2022 07:09:55 -0400 X-mail-filterd: {"version":"1.4.3", "queueID":"4LC8gR439Qz7t7d", "contextId":"e783ce7d-ad54-4762-935a-235d331bce01"} Original-Received: from outgoing-mail.laposte.net (localhost.localdomain [127.0.0.1]) by mlpnf0102.laposte.net (SMTP Server) with ESMTP id 4LC8gR439Qz7t7d for ; Tue, 31 May 2022 13:09:39 +0200 (CEST) X-mail-filterd: {"version":"1.4.3", "queueID":"4LC8gR37yzz7t7b", "contextId":"8466093c-ebf5-4c9c-8b13-3130e6cda22f"} X-lpn-mailing: LEGIT X-lpn-spamrating: 50 X-lpn-spamlevel: not-spam Original-Received: from [192.168.29.56] (82-64-20-182.subs.proxad.net [82.64.20.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mlpnf0102.laposte.net (SMTP Server) with ESMTPSA id 4LC8gR37yzz7t7b for ; Tue, 31 May 2022 13:09:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=laposte.net; s=lpn-wlmd; t=1653995381; bh=YSLPmL+b+dXC9RtQwX7J1TbRJ19DfK7isCBJrpCa35c=; h=Message-ID:Subject:From:To:Date:Content-Type:MIME-Version; b=WDDVyeFlVr2F8Hxsl+xkjLBUcRSCsdwafwqBN18Fg5RW+p3zByBPAh+vg3cayDRLtbTQuu/8lSn9Dtb6rBttJDhGwX5fRvfHJEypG/zllw8pfNBNgUG6DKIjbLTx2loYt3fEbuDulpHjMtL7/7mi8RP1egluWzK720Rjp2zmniN8Hcr9SnGksZ+0xf4W/8Oa/Lq9LOTdx4mocdF3OgAtO32tTSdxZDimoY2prLeNFIwcBYsEMU13wWH7I0nOXmZR0dKZtZyYXnWP7p7qbn/iIDvl8nHFCbYtrcBylVLXtjKUyZKb2VVohCQMadUg9YOqvKgffMuWUck/vFK87HkBkQ==; Received-SPF: pass client-ip=160.92.124.102; envelope-from=frederic.giquel@laposte.net; helo=smtp-outgoing-1801.laposte.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:233425 Archived-At: --=-LE+vfNM1KAgPBsk8Pp4K Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, While trying to understand the reason for slow completion with lsp-mode in some situations, I noticed that the language server response is split into 64k chunks despite setting `read-process-output-max' to 1M (as recommended by lsp-mode maintainers). The problem is not lsp-mode specific and can be reproduced without additionnal packages by using the two scripts attached to this email: - "write-server.py" is the server: it waits for a number (requested_size) on its input and write a string of "requested_size" bytes on its output. The file must be executable and located in the PATH. - "real-pipe-size.el" launch "write-server.py" and communicate with the process using different values for `read-process-output-max'. It prints the maximum chunk size used for each value. On my system, I got the following output executing 'emacs -Q --script ~/Projets/emacs-pipe/real-pipe-size.el': Real pipe size (with read-process-output-max =3D 4096): 4096 Real pipe size (with read-process-output-max =3D 32768): 32768 Real pipe size (with read-process-output-max =3D 65536): 65536 Real pipe size (with read-process-output-max =3D 65537): 65536 Real pipe size (with read-process-output-max =3D 204800): 65536 Real pipe size (with read-process-output-max =3D 1048576): 65536 Real pipe size (with read-process-output-max =3D 2097152): 65536 This 64k limit is GNU/Linux specific and is explained in "Pipe capacity" section of pipe(7) manpage and can be changed calling `fcntl'. So I added a call to `fcntl' for `F_SETPIPE_SZ' in `create_process' function just after other uses of `fcntl' (I'm not really sure if it's the right place). The patch is attached to this email. With the patch, the limit is now the value of `/proc/sys/fs/pipe-max- size' (which default to 1M and can be change by users with the CAP_SYS_RESOURCE capability). Output of 'emacs -Q --script ~/Projets/emacs-pipe/real-pipe-size.el' is now: Real pipe size (with read-process-output-max =3D 4096): 4096 Real pipe size (with read-process-output-max =3D 32768): 32768 Real pipe size (with read-process-output-max =3D 65536): 65536 Real pipe size (with read-process-output-max =3D 65537): 65537 Real pipe size (with read-process-output-max =3D 204800): 204800 Real pipe size (with read-process-output-max =3D 1048576): 1048576 Real pipe size (with read-process-output-max =3D 2097152): 65536 And my initial problem of slow completion with lsp-mode is fixed. P.S. I signed the FSF copyright assignment agreement for Emacs last year. In GNU Emacs 28.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0) of 2022-05-30 built on guiriden Repository revision: 5a223c7f2ef4c31abbd46367b6ea83cd19d30aa7 Repository branch: heads/emacs-28.1 Windowing system distributor 'The X.Org Foundation', version 11.0.12201000 System Description: Debian GNU/Linux bookworm/sid Configured features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB Important settings: value of $LANG: fr_FR.UTF-8 value of $XMODIFIERS: @im=3Dibus locale-coding-system: utf-8-unix Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t eldoc-mode: t show-paren-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t indent-tabs-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail rmail-loaddefs auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json map text-property-search time-date subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl- loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail- utils iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x- dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite emoji-zwj charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button loaddefs faces cus-face macroexp files window text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads dbusbind inotify lcms2 dynamic-setting system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 50603 6602) (symbols 48 6632 1) (strings 32 18327 1513) (string-bytes 1 607344) (vectors 16 13715) (vector-slots 8 184638 11282) (floats 8 21 47) (intervals 56 195 0) (buffers 992 10)) --=-LE+vfNM1KAgPBsk8Pp4K Content-Disposition: attachment; filename="write-server.py" Content-Type: text/x-python3; name="write-server.py"; charset="UTF-8" Content-Transfer-Encoding: base64 IyEgL3Vzci9iaW4vZW52IHB5dGhvbjMKCmltcG9ydCBzeXMKCndoaWxlIFRydWU6CiAgICByZXF1 ZXN0ZWRfc2l6ZSA9IGludChzeXMuc3RkaW4ucmVhZGxpbmUoKSkKICAgIHdyaXRlX3NpemUgPSBz eXMuc3Rkb3V0LndyaXRlKCJBIiAqIHJlcXVlc3RlZF9zaXplKQogICAgc3lzLnN0ZG91dC5mbHVz aCgpCg== --=-LE+vfNM1KAgPBsk8Pp4K Content-Disposition: attachment; filename="real-pipe-size.el" Content-Type: text/x-emacs-lisp; name="real-pipe-size.el"; charset="UTF-8" Content-Transfer-Encoding: base64 KGRlZnZhciBzaG93LWRldGFpbHMgbmlsKQoKOzsgVmFsdWVzIHRvIHVzZSBmb3IgYHJlYWQtcHJv Y2Vzcy1vdXRwdXQtbWF4JwooZGVmdmFyIHJlYWQtcHJvY2Vzcy1vdXRwdXQtbWF4LWxpc3QKICBg KAogICAgNDA5NiAgICAgICAgICAgICAgIDs7IGRlZmF1bHQgdmFsdWUKICAgICwoKiAzMiAxMDI0 KSAgICAgICA7OyAzMmsKICAgICwoKiA2NCAxMDI0KSAgICAgICA7OyA2NGsKICAgICwoKyAxICgq IDY0IDEwMjQpKSA7OyA2NGsgKyAxCiAgICAsKCogMjAwIDEwMjQpICAgICAgOzsgMjAwawogICAg LCgqIDEwMjQgMTAyNCkgICAgIDs7IDFNCiAgICAsKCogMiAxMDI0IDEwMjQpICAgOzsgMk0KICAg ICkpCgooZGVmdW4gZmlsdGVyLXNldC1tYXgtY2h1bmstc2l6ZSAocHJvYyBzdHJpbmcpCiAgKHdo ZW4gc2hvdy1kZXRhaWxzCiAgICAobWVzc2FnZSAiLi4uIGNodW5rIHNpemU6ICVzIiAobGVuZ3Ro IHN0cmluZykpKQogIChzZXRxIG1heC1jaHVuay1zaXplIChtYXggKGxlbmd0aCBzdHJpbmcpIG1h eC1jaHVuay1zaXplKSkpCgooZGVmdW4gY2FsY3VsYXRlLXJlYWwtcGlwZS1zaXplIChyZXF1ZXN0 ZWQtcGlwZS1zaXplKQogIChsZXQqICgocmVhZC1wcm9jZXNzLW91dHB1dC1tYXggcmVxdWVzdGVk LXBpcGUtc2l6ZSkKICAgICAgICAgKG1heC1jaHVuay1zaXplIDApCiAgICAgICAgIChwcm9jICht YWtlLXByb2Nlc3MKICAgICAgICAgICAgICAgIDpuYW1lICJjYWxjdWxhdGUtcGlwZS1zaXplIgog ICAgICAgICAgICAgICAgOmJ1ZmZlciBuaWwKICAgICAgICAgICAgICAgIDpjb21tYW5kICcoIndy aXRlLXNlcnZlci5weSIpCiAgICAgICAgICAgICAgICA6Y29ubmVjdGlvbi10eXBlICdwaXBlCiAg ICAgICAgICAgICAgICA6ZmlsdGVyICdmaWx0ZXItc2V0LW1heC1jaHVuay1zaXplKSkpCiAgICAo d2hlbiBzaG93LWRldGFpbHMKICAgICAgKG1lc3NhZ2UgIj09IENhbGN1bGF0aW5nIHJlYWwgcGlw ZSBzaXplIHdpdGggcmVhZC1wcm9jZXNzLW91dHB1dC1tYXggPSAlcyA9PSIgcmVhZC1wcm9jZXNz LW91dHB1dC1tYXgpKQogICAgKHByb2Nlc3Mtc2VuZC1zdHJpbmcgImNhbGN1bGF0ZS1waXBlLXNp emUiIChjb25jYXQgKG51bWJlci10by1zdHJpbmcgcmVxdWVzdGVkLXBpcGUtc2l6ZSkgIlxuIikp CiAgICAoYWNjZXB0LXByb2Nlc3Mtb3V0cHV0IHByb2MpCiAgICAoZGVsZXRlLXByb2Nlc3MgcHJv YykKICAgIG1heC1jaHVuay1zaXplKSkKCihkb2xpc3QgKHBpcGUtc2l6ZSByZWFkLXByb2Nlc3Mt b3V0cHV0LW1heC1saXN0KQogIChtZXNzYWdlICJSZWFsIHBpcGUgc2l6ZSAod2l0aCByZWFkLXBy b2Nlc3Mtb3V0cHV0LW1heCA9ICVzKTogJXMiIHBpcGUtc2l6ZSAoY2FsY3VsYXRlLXJlYWwtcGlw ZS1zaXplIHBpcGUtc2l6ZSkpKQo= --=-LE+vfNM1KAgPBsk8Pp4K Content-Disposition: attachment; filename="pipe-size.patch" Content-Type: text/x-patch; name="pipe-size.patch"; charset="UTF-8" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3NyYy9wcm9jZXNzLmMgYi9zcmMvcHJvY2Vzcy5jCmluZGV4IDhiNTg3YWFh NGUuLjcwMzM1MmU1OWUgMTAwNjQ0Ci0tLSBhL3NyYy9wcm9jZXNzLmMKKysrIGIvc3JjL3Byb2Nl c3MuYwpAQCAtMjE1NSw2ICsyMTU1LDEwIEBAIGNyZWF0ZV9wcm9jZXNzIChMaXNwX09iamVjdCBw cm9jZXNzLCBjaGFyICoqbmV3X2FyZ3YsIExpc3BfT2JqZWN0IGN1cnJlbnRfZGlyKQogICBmY250 bCAoaW5jaGFubmVsLCBGX1NFVEZMLCBPX05PTkJMT0NLKTsKICAgZmNudGwgKG91dGNoYW5uZWws IEZfU0VURkwsIE9fTk9OQkxPQ0spOwogCisjaWYgZGVmaW5lZChHTlVfTElOVVgpICYmIGRlZmlu ZWQoRl9TRVRQSVBFX1NaKQorICBmY250bCAoaW5jaGFubmVsLCBGX1NFVFBJUEVfU1osIHJlYWRf cHJvY2Vzc19vdXRwdXRfbWF4KTsKKyNlbmRpZgorCiAgIC8qIFJlY29yZCB0aGlzIGFzIGFuIGFj dGl2ZSBwcm9jZXNzLCB3aXRoIGl0cyBjaGFubmVscy4gICovCiAgIGVhc3NlcnQgKDAgPD0gaW5j aGFubmVsICYmIGluY2hhbm5lbCA8IEZEX1NFVFNJWkUpOwogICBjaGFuX3Byb2Nlc3NbaW5jaGFu bmVsXSA9IHByb2Nlc3M7CkBAIC04NTcxLDcgKzg1NzUsMTAgQEAgc3ltc19vZl9wcm9jZXNzICh2 b2lkKQogICBERUZWQVJfSU5UICgicmVhZC1wcm9jZXNzLW91dHB1dC1tYXgiLCByZWFkX3Byb2Nl c3Nfb3V0cHV0X21heCwKIAkgICAgICBkb2M6IC8qIE1heGltdW0gbnVtYmVyIG9mIGJ5dGVzIHRv IHJlYWQgZnJvbSBzdWJwcm9jZXNzIGluIGEgc2luZ2xlIGNodW5rLgogRW5sYXJnZSB0aGUgdmFs dWUgb25seSBpZiB0aGUgc3VicHJvY2VzcyBnZW5lcmF0ZXMgdmVyeSBsYXJnZSAobWVnYWJ5dGVz KQotYW1vdW50cyBvZiBkYXRhIGluIG9uZSBnby4gICovKTsKK2Ftb3VudHMgb2YgZGF0YSBpbiBv bmUgZ28uCisKK09uIEdOVS9MaW51eCBzeXN0ZW0sIHRoZSB2YWx1ZSBzaG91bGQgbm90IGV4Y2Vl ZAorYC9wcm9jL3N5cy9mcy9waXBlLW1heC1zaXplJy4gU2VlIHBpcGUoNykgbWFucGFnZSBmb3Ig ZGV0YWlscy4gKi8pOwogICByZWFkX3Byb2Nlc3Nfb3V0cHV0X21heCA9IDQwOTY7CiAKICAgREVG U1lNIChRaW50ZXJuYWxfZGVmYXVsdF9pbnRlcnJ1cHRfcHJvY2VzcywK --=-LE+vfNM1KAgPBsk8Pp4K--