From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: "J.P." Newsgroups: gmane.emacs.bugs Subject: bug#54458: 27.2; erc-dcc-get: Re-entering top level after C stack overflow Date: Mon, 28 Mar 2022 05:08:56 -0700 Message-ID: <87r16m46uf.fsf@neverwas.me> References: <78459EAB-314B-4122-8E3B-7F82685D0DBA@acm.org> <83a6da9vm8.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20402"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux) Cc: Mattias =?UTF-8?Q?Engdeg=C3=A5rd?= , fernandodemorais.jf@gmail.com, bandali@gnu.org, 54458@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Mar 28 14:10:51 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1nYoCw-00051S-Hk for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 28 Mar 2022 14:10:50 +0200 Original-Received: from localhost ([::1]:60648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nYoCu-0003Gm-FU for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 28 Mar 2022 08:10:48 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:38312) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nYoCA-0003Ga-DO for bug-gnu-emacs@gnu.org; Mon, 28 Mar 2022 08:10:02 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:35301) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nYoCA-0001WR-4U for bug-gnu-emacs@gnu.org; Mon, 28 Mar 2022 08:10:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nYoC9-0005km-Tf for bug-gnu-emacs@gnu.org; Mon, 28 Mar 2022 08:10:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: "J.P." Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 28 Mar 2022 12:10:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54458 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: moreinfo Original-Received: via spool by 54458-submit@debbugs.gnu.org id=B54458.164846934822046 (code B ref 54458); Mon, 28 Mar 2022 12:10:01 +0000 Original-Received: (at 54458) by debbugs.gnu.org; 28 Mar 2022 12:09:08 +0000 Original-Received: from localhost ([127.0.0.1]:57431 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nYoBH-0005jV-SO for submit@debbugs.gnu.org; Mon, 28 Mar 2022 08:09:08 -0400 Original-Received: from mail-108-mta14.mxroute.com ([136.175.108.14]:35151) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nYoBF-0005iw-Iu for 54458@debbugs.gnu.org; Mon, 28 Mar 2022 08:09:06 -0400 Original-Received: from filter006.mxroute.com ([140.82.40.27] 140.82.40.27.vultrusercontent.com) (Authenticated sender: mN4UYu2MZsgR) by mail-108-mta14.mxroute.com (ZoneMTA) with ESMTPSA id 17fd06dee6d000fe85.002 for <54458@debbugs.gnu.org> (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256); Mon, 28 Mar 2022 12:08:59 +0000 X-Zone-Loop: 4a8697d57cdc3eae12d17039e1b5659c714b23a14602 X-Originating-IP: [140.82.40.27] DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=neverwas.me ; s=x; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:References: Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=jvmQaFnw/m0cmyJrrQDlHVOrJHyTACu5e2YpVhMAxRc=; b=JA8IRpeQK5y76lDVetnAIVrXtQ TsPfTYkAa4tJGbR2ruD4a1VV7oLEyd6UdeJrCg/I7NndkwCahgNlWxiXIS4/KUHuzdB24v+yexDM7 FRYk20Vu9dDa1ApXQNb6YpRbzt5yctPNWq67WjO2/02b52xY6s6f3u6yO1YbcUnbfQChQtgb1jECx FiTnCmPhDmSFxlQeOGWphcMcFhISS/kKDn9vU+i3ZTVK2NJ7G9qqJ43sHGpcjzh+syU+Yi0BFpO8E 2rA3ItFAkE879/aQ+uS8IjXUZR9ng8N0KuYYPadGfxFW0VwfY/ToQH4ume1Il6ob9e+pE9M9z+9Tr uHvJdiiA==; In-Reply-To: <83a6da9vm8.fsf@gnu.org> (Eli Zaretskii's message of "Mon, 28 Mar 2022 14:14:55 +0300") X-AuthUser: masked@neverwas.me X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:229008 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Mattias (and Eli), Mattias Engdeg=C3=A5rd writes: > 27 mars 2022 kl. 22.54 skrev Mattias Engdeg=C3=A5rd : > >> Not sure where this happens but it looks like it might be erc-dcc-send-f= ilter. > > Presumably we should add a note to the documentation of process filter > functions that they shouldn't be used to send more data to the process. (= Not > sure what to recommend, an idle timer maybe?) As you highlighted up thread, inhibited sends appeal to `wait_reading_process_output' to try and shake something loose. I agree such occasions seem likeliest to trigger "unexpected" filter nesting. As far as best practices go, I'm not sure how a successful request-response dialog can happen without participants being able to react in a timely fashion. If the main worry is stack growth, then perhaps scheduling something on the event loop with a timer (as you say) makes the most sense. I actually tried that (via `run-at-time') in the tinkering detailed below but still managed to incur "error running timer" messages that referred to "excessive variable binding." Should I have used something in the "idle" department instead? The approach that seemed to "work" mimics the one relied on by ERC's main client filter, which takes pains to ensure overlapping calls merely stash the input and bail (via `erc-server-processing-p'). Perhaps a synchronization primitive especially suited to process filters would make more sense? (No idea.) > I'm not going to fix this because I don't know ERC very well and wouldn't= be > able to test it sufficiently, but our ERC maintainers do and can! You're very generous, and your expertise is much appreciated. (But as Eli says, "hopefully".) . . . Hi Fernando, I've managed to trigger behavior (somewhat) resembling what you've described. It's quite possible, even likely, that this is total baloney, but please humor me anyway this one round (unless you spot something totally egregious, that is). The basic "hypothesis" seems to comport with Mattias's analysis. It posits that the peer you're connecting to is misbehaving and engaged in some combination of: 1. not reading frequently enough amid sends 2. being too stingy with its read buffer I'd be great if you could confirm this suspicion by checking if the "window" portion of the TCP ACK segments containing actual payloads go to zero after a spell. This can be done with tcpdump or wireshark. A well behaved peer respecting the protocol should normally have nonzero windows throughout. On the client side (Emacs master this time), here is what I observe after following the steps enumerated further down: There appears to be another buffer of around 64KB that needs filling before I (the receiver) encounter the error, which in my case is a "variable binding depth exceeds max-specpdl-size" message along with an unresponsive UI. For me, this happens without fail at around 800MB after the TCP window goes to 0 (at around 100MB). Strangely, increasing the value of `max-specpdl-size' doesn't change things perceptively. Anyway, the file-writing operation continues for around 200MB and eventually peters out. But the connection only dies after the sender closes it normally (having sent everything). The IRC connection of course PINGs out and is severed by the IRC server. The Emacs receiver process eventually recovers responsiveness (if you wait long enough). These are the steps I followed. They require two emacs -Q instances, a server, and the attached script: 1. connect to the server and make sure the sender can /whois the receiver (have them join the same #chan if necessary) =20 2. start the script: python ./script.py ./some_large_file.bin misbehave =20 3. on the sender: /msg RecvNick ^ADCC SEND some_large_file.bin 2130706433 9899 1234567890= ^A where 1234567890 is the size of the file in bytes and ^A is an actual control char =20 4. on the receiver: /dcc get SendNick some_large_file.bin As mentioned earlier, I've attached a crude experiment (patch) that just records whether we're currently sending (so receipts can be skipped when the flag is set). I used the process plist for now, but erc-dcc does keep a global context-state object called `erc-dcc-entry-data', which I suppose may be more fitting. The idea is to roll with the punches from a pathological peer but also (of course) interoperate correctly with an obedient, protocol-abiding one. Normal sender - Sends 1405135128 bytes - Sees 343051 reports Aberrant sender - Sends 1405135128 bytes - Ignores 238662 + unknown reports Let me know if anything needs clarifying. And thanks for your patience! --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=serve.py Content-Transfer-Encoding: base64 IiIiSG9zdGlsZSBEQ0MtU0VORCBlbmRwb2ludAoKVXNhZ2U6IHB5dGhvbiB0aGlzX3NjcmlwdC5w eSAuL2Jsb2IuYmluIFthY3RpdmVfZmxhZ10KCldpdGggYGBhY3RpdmVfZmxhZ2BgLCBkb24ndCB3 YWl0IGZvciByZWFkIHJlY2VpcHRzIGJlZm9yZSBzZW5kaW5nIHRoZQpuZXh0IGh1bmsuCgoiIiIK aW1wb3J0IHN5cwppbXBvcnQgc29ja2V0CmltcG9ydCBhc3luY2lvCgpmcm9tIHBhdGhsaWIgaW1w b3J0IFBhdGgKCgpjbGFzcyBPbkNvbm5lY3Q6CiAgICBkZWYgX19pbml0X18oc2VsZiwgZmlsZSwg bm9fcmVjdj1GYWxzZSk6CiAgICAgICAgc2VsZi5maWxlID0gUGF0aChmaWxlKQogICAgICAgIHNl bGYuc2tpcCA9IGJvb2wobm9fcmVjdikKICAgICAgICBwcmludChmIlNlbmRpbmcge2ZpbGUhcn0g KHtzZWxmLmZpbGUuc3RhdCgpLnN0X3NpemV9IGJ5dGVzKSIpCiAgICAgICAgcHJpbnQoIkZpcmVo b3NlIG1vZGU6IiwgIm9uIiBpZiBzZWxmLnNraXAgZWxzZSAib2ZmIikKCiAgICBhc3luYyBkZWYg cmVhZChzZWxmKSAtPiBpbnQ6CiAgICAgICAgZGF0YSA9IGF3YWl0IHNlbGYucmVhZGVyLnJlYWQo MTAyNCkKICAgICAgICBkbGVuID0gbGVuKGRhdGEpCiAgICAgICAgcHJpbnQoIi4iIGlmIGRsZW4g PT0gNCBlbHNlIGYiW3tkbGVufV0iLCBlbmQ9IiIsIGZsdXNoPVRydWUpCiAgICAgICAgcmV0dXJu IGRsZW4KCiAgICBhc3luYyBkZWYgaGFuZGxlKHNlbGYpOgogICAgICAgIHNlbnQgPSByZWNlaXZl ZCA9IDAKICAgICAgICBwcmludChmIkNvbm5lY3Rpb24gZnJvbSB7c2VsZi53cml0ZXIuZ2V0X2V4 dHJhX2luZm8oJ3BlZXJuYW1lJykhcn0iKQoKICAgICAgICB3aXRoIHNlbGYuZmlsZS5vcGVuKCJy YiIpIGFzIGY6CiAgICAgICAgICAgIHdoaWxlIGNodW5rIDo9IGYucmVhZCgzMjc2OCk6CiAgICAg ICAgICAgICAgICBzZWxmLndyaXRlci53cml0ZShjaHVuaykKICAgICAgICAgICAgICAgIGF3YWl0 IHNlbGYud3JpdGVyLmRyYWluKCkKICAgICAgICAgICAgICAgIHNlbnQgKz0gbGVuKGNodW5rKQog ICAgICAgICAgICAgICAgaWYgc2VsZi5za2lwOgogICAgICAgICAgICAgICAgICAgIGNvbnRpbnVl CiAgICAgICAgICAgICAgICByZWNlaXZlZCArPSBhd2FpdCBzZWxmLnJlYWQoKQoKICAgICAgICAg ICAgdHJ5OgogICAgICAgICAgICAgICAgd2hpbGUgZyA6PSBhd2FpdCBhc3luY2lvLndhaXRfZm9y KHNlbGYucmVhZCgpLCB0aW1lb3V0PTEpOgogICAgICAgICAgICAgICAgICAgIHJlY2VpdmVkICs9 IGcKICAgICAgICAgICAgZXhjZXB0IGFzeW5jaW8uVGltZW91dEVycm9yOgogICAgICAgICAgICAg ICAgcGFzcwogICAgICAgICAgICBwcmludCgiXG5TZW50ICVkIGJ5dGVzIiAlIHNlbnQpCiAgICAg ICAgICAgIHByaW50KCJTYXcgJWQgcmVwb3J0cyIgJSAocmVjZWl2ZWQgLy8gNCkpCiAgICAgICAg ICAgIHNlbGYud3JpdGVyLmNsb3NlKCkKICAgICAgICAgICAgYXdhaXQgc2VsZi53cml0ZXIud2Fp dF9jbG9zZWQoKQoKICAgIGRlZiBfX2NhbGxfXyhzZWxmLCByZWFkZXIsIHdyaXRlcik6CiAgICAg ICAgc2VsZi5yZWFkZXIgPSByZWFkZXIKICAgICAgICBzZWxmLndyaXRlciA9IHdyaXRlcgogICAg ICAgIHJldHVybiBzZWxmLmhhbmRsZSgpCgoKYXN5bmMgZGVmIG1haW4oaGFuZGxlcjogT25Db25u ZWN0KToKICAgIHNvY2sgPSBzb2NrZXQuc29ja2V0KHNvY2tldC5BRl9JTkVULCBzb2NrZXQuU09D S19TVFJFQU0pCiAgICBzb2NrLnNldHNvY2tvcHQoc29ja2V0LlNPTF9TT0NLRVQsIHNvY2tldC5T T19SRVVTRUFERFIsIFRydWUpCiAgICBzb2NrLnNldHNvY2tvcHQoc29ja2V0LlNPTF9TT0NLRVQs IHNvY2tldC5TT19SQ1ZCVUYsIDEyOCkKICAgIHNvY2suYmluZCgoIjEyNy4wLjAuMSIsIDk4OTkp KQogICAgc2VydmVyID0gYXdhaXQgYXN5bmNpby5zdGFydF9zZXJ2ZXIoaGFuZGxlciwgc29jaz1z b2NrKQogICAgcHJpbnQoZiJTZXJ2aW5nIG9uIHtzZXJ2ZXIuc29ja2V0c1swXS5nZXRzb2NrbmFt ZSgpfSIpCgogICAgYXN5bmMgd2l0aCBzZXJ2ZXI6CiAgICAgICAgYXdhaXQgc2VydmVyLnNlcnZl X2ZvcmV2ZXIoKQoKCmlmIF9fbmFtZV9fID09ICJfX21haW5fXyI6CiAgICB0cnk6CiAgICAgICAg YXN5bmNpby5ydW4obWFpbihPbkNvbm5lY3QoKnN5cy5hcmd2WzE6XSkpKQogICAgZXhjZXB0IEtl eWJvYXJkSW50ZXJydXB0OgogICAgICAgIHByaW50KCkK --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-EXPERIMENT-regulate-ACK-updates-in-erc-dcc-get-filte.patch >From 7f9a7fe1a1ea9208841a4ec600ec25e1a464b71d Mon Sep 17 00:00:00 2001 From: "F. Jason Park" Date: Mon, 28 Mar 2022 02:24:43 -0700 Subject: [PATCH] [EXPERIMENT] regulate ACK updates in erc-dcc-get-filter * lisp/erc/erc-dcc.el (erc-dcc-get-filter): Don't bother sending a "received so far" receipt if another attempt is in progress. --- lisp/erc/erc-dcc.el | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lisp/erc/erc-dcc.el b/lisp/erc/erc-dcc.el index cc4143bfa2..aaac2277bf 100644 --- a/lisp/erc/erc-dcc.el +++ b/lisp/erc/erc-dcc.el @@ -986,9 +986,10 @@ erc-dcc-get-filter 'dcc-get-file-too-long ?f (file-name-nondirectory (buffer-name))) (delete-process proc)) - (t - (process-send-string - proc (erc-pack-int received-bytes))))))) + ((not (process-get proc :sending)) + (process-put proc :sending t) ; should maybe use `erc-dcc-entry-data' + (process-send-string proc (erc-pack-int received-bytes)) + (process-put proc :sending nil)))))) (defun erc-dcc-get-sentinel (proc _event) -- 2.35.1 --=-=-=--