From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Troy Hinckley Newsgroups: gmane.emacs.bugs Subject: bug#71896: shell-resync-dirs hang Date: Mon, 1 Jul 2024 21:22:35 -0500 Message-ID: <83b035dd-1ba4-4016-8ba4-cf9bde800175@Spark> References: <03321175-1a92-4b82-a0cc-d6977b6a733a@Spark> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="66836470_6eb5bd4_524" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="17474"; mail-complaints-to="usenet@ciao.gmane.io" To: 71896@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jul 02 07:17:21 2024 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sOVtJ-0004LS-Jl for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 02 Jul 2024 07:17:21 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sOVt3-0007Gk-C7; Tue, 02 Jul 2024 01:17:05 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOVt1-0007Fr-P9 for bug-gnu-emacs@gnu.org; Tue, 02 Jul 2024 01:17:03 -0400 Original-Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sOVt1-0003Of-G8 for bug-gnu-emacs@gnu.org; Tue, 02 Jul 2024 01:17:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1sOVt2-0004Ib-CR for bug-gnu-emacs@gnu.org; Tue, 02 Jul 2024 01:17:04 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Troy Hinckley Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Tue, 02 Jul 2024 05:17:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 71896 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.171989739416433 (code B ref -1); Tue, 02 Jul 2024 05:17:04 +0000 Original-Received: (at submit) by debbugs.gnu.org; 2 Jul 2024 05:16:34 +0000 Original-Received: from localhost ([127.0.0.1]:35518 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sOVsX-0004Gt-0X for submit@debbugs.gnu.org; Tue, 02 Jul 2024 01:16:33 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:42064) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1sOTAc-0008BP-Fs for submit@debbugs.gnu.org; Mon, 01 Jul 2024 22:23:04 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOTAZ-0002EC-BW for bug-gnu-emacs@gnu.org; Mon, 01 Jul 2024 22:23:01 -0400 Original-Received: from sender4-op-o15.zoho.com ([136.143.188.15]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sOTAU-0005oe-Lp for bug-gnu-emacs@gnu.org; Mon, 01 Jul 2024 22:22:57 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1719886968; cv=none; d=zohomail.com; s=zohoarc; b=MlqksOvdRHBJzUKaxKrnzD2Dx85k/sRmYVRwGwWUhGhmC1NBTB6wXlryDxYyMBf+pXOd4jTkfWCkiN91E0tVkKKJnira1CuPKRmkyojS7Ay9vUfsIFCFZvpRi0Kp3Yy1QhjocK/QYGnvcEuk23rI52aoDH1tEy3towZDq2LitMA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1719886968; h=Content-Type:Date:Date:From:From:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=Z1pWzUqgNYF1OqotZ6NaxNjWDOYGh/pXgX+8Wah7kT4=; b=GH5jhtKLnskAnsxv7InxVL7IaM026NRrqbeZOzgjoGhAQcYqaSnwwtfHY8nNsI4+TYS9/rgDj7FsdpVzQDNXyNx+NWrmXSBpWdhAOt7MLeNVpV/vzkwDCUjGEe7ygilxyUdwSgaf/cmeInO0Qt6FzseXrtaF8H7hmKWi2NsiIVc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=dabrev.com; spf=pass smtp.mailfrom=troyhinckley@dabrev.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1719886968; s=zoho; d=dabrev.com; i=troyhinckley@dabrev.com; h=Date:Date:From:From:To:To:Message-ID:References:Subject:Subject:MIME-Version:Content-Type:Message-Id:Reply-To:Cc; bh=Z1pWzUqgNYF1OqotZ6NaxNjWDOYGh/pXgX+8Wah7kT4=; b=hiZkkkSkb5CvaCN6RVSa6gZhMrY/JGzJI1x8gg0lD5SbVrnGVLmxyZB4K9U3ggH3 /J1Ww7LmyOMRx8VxFYvUTiiPeARKP/IW8yOATxJ8F9ZKQ6RUpLLbBfo77AgUBAorMtU K1tS8inGIhCjoYc5YwLhP6pOS8K2d6zqO+H0QRPE= Original-Received: by mx.zohomail.com with SMTPS id 171988696629677.4677691426291; Mon, 1 Jul 2024 19:22:46 -0700 (PDT) X-Readdle-Message-ID: 83b035dd-1ba4-4016-8ba4-cf9bde800175@Spark X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.15; envelope-from=troyhinckley@dabrev.com; helo=sender4-op-o15.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 02 Jul 2024 01:16:24 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.bugs:288276 Archived-At: --66836470_6eb5bd4_524 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline This is related to =2359804 and =2354384. I will occasionally (about 30% of the time) see hangs when running shell-= resync-dirs in Emacs 29. I tracked this down to two issues: =46irst is in shell-eval-command. =46or Emacs 29 this function was added = to fix =2354384. It has this section in the code: =60=60=60 ;; Wait until we get a prompt (which will be a line without =C2=A0;; a newline). This is far from fool-proof -- if something =C2=A0;; outputs incomplete data and then sleeps, we'll think =C2=A0;; we've received the prompt. =C2=A0(while (not (let* ((lines (string-lines result)) =C2=A0(last (car (last lines)))) =C2=A0(and (length> lines 0) =C2=A0(not (equal last =22=22)) =C2=A0(or (not prev) =C2=A0(not (equal last prev))) =C2=A0(setq prev last)))) =C2=A0(accept-process-output proc 0 100)) =60=60=60 Note that is says that is looking for =E2=80=9Ca line without a newline=E2= =80=9D to determine if we have reached the prompt. However this code does= not actually do that. If the result ends in a newline it will still term= inate the loop and not wait for more input. We can see that by the fact t= hat the following code evaluates to nil. =60=60=60 (let ((result =22dirs=5Cn=22) prev) =C2=A0(not (let* ((lines (string-lines result)) =C2=A0(last (car (last lines)))) =C2=A0(and (length> lines 0) =C2=A0(not (equal last =22=22)) =C2=A0(or (not prev) =C2=A0(not (equal last prev))) =C2=A0(setq prev last))))) =60=60=60 I am not sure what this code is supposed to do, but the issue arrises if = the process output sends anything to this function it will terminate and = not wait for more input. In my case the issue is that the shell is echoin= g the command followed by the result (comint-process-echoes). About 30% o= f the time these two lines get sent as part of two different outputs. Mea= ning the second line (the directory for shell-resync-dirs) does not get c= aptured and instead gets printed to the terminal. This leads us to the hang. The issue is this code in shell-resync-dirs: =60=60=60 (while dlsl =C2=A0(let ((newelt =22=22) =C2=A0tem1 tem2) =C2=A0(while newelt =C2=A0;; We need tem1 because we don't want to prepend =C2=A0;; =60comint-file-name-prefix' repeatedly into newelt via tem2. =C2=A0(setq tem1 (pop dlsl) =C2=A0tem2 (concat comint-file-name-prefix tem1 newelt)) =C2=A0(cond ((file-directory-p tem2) =C2=A0(push tem2 ds) =C2=A0(when (string=3D =22 =22 (car dlsl)) =C2=A0(pop dlsl)) =C2=A0(setq newelt nil)) =C2=A0(t =C2=A0(setq newelt (concat tem1 newelt))))))) =60=60=60 This loop can only terminate if tem2 is a valid directory. Otherwise it w= ill take the default case in the cond and loop forever. And since the bug= in shell-eval-command does not provide the directory when the process ou= tput is split, we get a hang. I believe both of these need to be fixed to properly fix the bug. =46or the shell-eval-command I don=E2=80=99t understand what that loop is= trying to do now, so I am not sure how to fix it without breaking its fu= nctionality. I would just use (string-suffix-p =E2=80=9C=5Cn=E2=80=9D res= ult) to check if the output ends in a newline, but the code is obviously = trying to do something more complex there. If we fix that issue then it will resolve the hang in shell-resync-dirs, = but I think that is just glossing over the problem. That functions should= never hang, no matter what output it get=E2=80=99s from the shell. My re= commendation would be to add =60(and dlsl newelt)=60 as the condition for= the inner while loop with newelt. That way if dlsl is empty, it will ter= minate the loop since there is nothing more to process. This fixed the is= sue for me. -Troy Hinckley --66836470_6eb5bd4_524 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
This is related to =2359804 and =2354384.

I will occasionally (about 30% of the time) see hangs when running shell-= resync-dirs in Emacs 29. I tracked this down to two issues:


=46irst is in shell-eval-command. =46or Emacs 29 this function was added = to fix =2354384. It has this section in the code:

=60=60=60
;; Wait until we get a prompt (which will be a line without
&=23160;;; a newline). This is far from fool-proof -- if something
&=23160;;; outputs incomplete data and then sleeps, we'll think
&=23160;;; we've received the prompt.
&=23160;(while (not (let* ((lines (string-lines result))
&=23160;(last (car (last lines))))
&=23160;(and (length> lines 0)
&=23160;(not (equal last =22=22))
&=23160;(or (not prev)
&=23160;(not (equal last prev)))
&=23160;(setq prev last))))
&=23160;(accept-process-output proc 0 100))
=60=60=60

Note that is says that is looking for =E2=80=9Ca line without a newline=E2= =80=9D to determine if we have reached the prompt. However this code does= not actually do that. If the result ends in a newline it will still term= inate the loop and not wait for more input. We can see that by the fact t= hat the following code evaluates to nil.

=60=60=60
(let ((result =22dirs=5Cn=22) prev)
&=23160;(not (let* ((lines (string-lines result))
&=23160;(last (car (last lines))))
&=23160;(and (length> lines 0)
&=23160;(not (equal last =22=22))
&=23160;(or (not prev)
&=23160;(not (equal last prev)))
&=23160;(setq prev last)))))
=60=60=60

I am not sure what this code is supposed to do, but the issue arrises if = the process output sends anything to this function it will terminate and = not wait for more input. In my case the issue is that the shell is echoin= g the command followed by the result (comint-process-echoes). About 30% o= f the time these two lines get sent as part of two different outputs. Mea= ning the second line (the directory for shell-resync-dirs) does not get c= aptured and instead gets printed to the terminal.


This leads us to the hang. The issue is this code in shell-resync-dirs:
=60=60=60
(while dlsl
&=23160;(let ((newelt =22=22)
&=23160;tem1 tem2)
&=23160;(while newelt
&=23160;;; We need tem1 because we don't want to prepend
&=23160;;; =60comint-file-name-prefix' repeatedly into newelt via tem2. &=23160;(setq tem1 (pop dlsl)
&=23160;tem2 (concat comint-file-name-prefix tem1 newelt))
&=23160;(cond ((file-directory-p tem2)
&=23160;(push tem2 ds)
&=23160;(when (string=3D =22 =22 (car dlsl))
&=23160;(pop dlsl))
&=23160;(setq newelt nil))
&=23160;(t
&=23160;(setq newelt (concat tem1 newelt)))))))
=60=60=60

This loop can only terminate if tem2 is a valid directory. Otherwise it w= ill take the default case in the cond and loop forever. And since the bug= in shell-eval-command does not provide the directory when the process ou= tput is split, we get a hang.

I believe both of these need to be fixed to properly fix the bug.

=46or the shell-eval-command I don=E2=80=99t understand what that loop is= trying to do now, so I am not sure how to fix it without breaking its fu= nctionality. I would just use (string-suffix-p =E2=80=9C=5Cn=E2=80=9D res= ult) to check if the output ends in a newline, but the code is obviously = trying to do something more complex there.

If we fix that issue then it will resolve the hang in shell-resync-dirs, = but I think that is just glossing over the problem. That functions should= never hang, no matter what output it get=E2=80=99s from the shell. My re= commendation would be to add =60(and dlsl newelt)=60 as the condition for= the inner while loop with newelt. That way if dlsl is empty, it will ter= minate the loop since there is nothing more to process. This fixed the is= sue for me.&=23160;

-Troy Hinckley
--66836470_6eb5bd4_524--