bug#54776: 28.1; shell-resync-dirs hangs

bug#54776: 28.1; shell-resync-dirs hangs

[Kevin K. Lin]

Hi,

Starting with

emacs -Q

I do

M-x shell
M-x dirs

and Emacs hangs.  C-g breaks the loop and puts me back
at the prompt.  FYI, I use /bin/tcsh, but the same
thing happens in other shells I tried (zsh, bash).
I'm on macOS; not sure the problem is Mac-specific.

I found bug #54384 on the bug tracker, which seems to
describe a similar problem in Emacs 29.  Wondering if
it's indeed the same issue, and if so whether the
solution would be backported to 28?

Thanks,
Kevin


In GNU Emacs 28.1 (build 1, x86_64-apple-darwin19.6.0)
 of 2022-04-04 built on iMac-Pro
System Description:  Mac OS X 10.15.7

Configured using:
 'configure --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/28.1/share/info/emacs
 --prefix=/usr/local/Cellar/emacs/28.1 --with-gnutls --without-x
 --with-xml2 --without-dbus --with-modules --without-ns
 --without-imagemagick --without-selinux'

Configured features:
ACL GMP GNUTLS JSON LIBXML2 MODULES NOTIFY KQUEUE PDUMPER THREADS ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(help-mode pp shadow sort mail-extr emacsbug message rmc puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg rfc6068 epg-config
gnus-util rmail tool-bar rmail-loaddefs auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json map
text-property-search time-date subr-x seq mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils shell
pcomplete comint regexp-opt ansi-color ring term/xterm xterm byte-opt gv
bytecomp byte-compile cconv iso-transl tooltip eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode tabulated-list
replace newcomment text-mode lisp-mode prog-mode register page tab-bar
menu-bar rfn-eshadow isearch easymenu timer select mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads kqueue
multi-tty make-network-process emacs)

Memory information:
((conses 16 60451 7936)
 (symbols 48 7359 3)
 (strings 32 20473 1785)
 (string-bytes 1 674989)
 (vectors 16 12526)
 (vector-slots 8 145293 8573)
 (floats 8 29 293)
 (intervals 56 220 54)
 (buffers 992 12))

bug#54776: 28.1; shell-resync-dirs hangs

[Lars Ingebrigtsen]

"Kevin K. Lin" <klin@math.arizona.edu> writes:

> I found bug #54384 on the bug tracker, which seems to
> describe a similar problem in Emacs 29.  Wondering if
> it's indeed the same issue, and if so whether the
> solution would be backported to 28?

It's the same issue, and we're only backporting fixes for regressions or
serious bugs to Emacs 28, and I don't think this counts as either, so it
won't be backported.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#54776: 28.1; shell-resync-dirs hangs

[Glenn Morris]

Lars Ingebrigtsen wrote:

> and we're only backporting fixes for regressions or serious bugs to
> Emacs 28, and I don't think this counts as either, so it won't be
> backported.

Are you sure this issue isn't due to the changes from
https://debbugs.gnu.org/23324, which were applied in Emacs 28.1 (which
would make it a regression from Emacs 27, no)?

bug#54776: 28.1; shell-resync-dirs hangs

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

Glenn Morris <rgm@gnu.org> writes:

> Are you sure this issue isn't due to the changes from
> https://debbugs.gnu.org/23324, which were applied in Emacs 28.1 (which
> would make it a regression from Emacs 27, no)?

Good catch.

Kevin, could you try reverting 90e65c826fa (or applying (in reverse) the
attached patch) and see whether that fixes the problem?  If so, we could
revert the patch for Emacs 28.2.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: resync.patch --]
[-- Type: text/x-diff, Size: 3042 bytes --]

commit 90e65c826fab2092ad2099d7763538194c93e021
Author:     Noah Friedman <friedman@splode.com>
AuthorDate: Wed Aug 19 15:59:59 2020 +0200
Commit:     Lars Ingebrigtsen <larsi@gnus.org>
CommitDate: Wed Aug 19 15:59:59 2020 +0200

    Make shell-resync-dirs handle whitespace in directory names
    
    * lisp/shell.el (shell-resync-dirs): Correctly handle
    whitespace in directory names (bug#23324).

diff --git a/lisp/shell.el b/lisp/shell.el
index 301a8cb083..9667dab2af 100644
--- a/lisp/shell.el
+++ b/lisp/shell.el
@@ -1035,25 +1035,41 @@ shell-resync-dirs
 	  (accept-process-output proc)
 	  (goto-char pt)))
       (goto-char pmark) (delete-char 1) ; remove the extra newline
-      ;; That's the dirlist. grab it & parse it.
-      (let* ((dl (buffer-substring (match-beginning 2) (1- (match-end 2))))
-	     (dl-len (length dl))
-	     (ds '())			; new dir stack
-	     (i 0))
-	(while (< i dl-len)
-	  ;; regexp = optional whitespace, (non-whitespace), optional whitespace
-	  (string-match "\\s *\\(\\S +\\)\\s *" dl i) ; pick off next dir
-	  (setq ds (cons (concat comint-file-name-prefix
-				 (substring dl (match-beginning 1)
-					    (match-end 1)))
-			 ds))
-	  (setq i (match-end 0)))
-	(let ((ds (nreverse ds)))
-	  (with-demoted-errors "Couldn't cd: %s"
-	    (shell-cd (car ds))
-	    (setq shell-dirstack (cdr ds)
-		  shell-last-dir (car shell-dirstack))
-	    (shell-dirstack-message)))))
+      ;; That's the dirlist.  Grab it & parse it.
+      (let* ((dls (buffer-substring-no-properties
+                   (match-beginning 0) (1- (match-end 0))))
+             (dlsl nil)
+             (pos 0)
+             (ds nil))
+        ;; Split the dirlist into whitespace and non-whitespace chunks.
+        ;; dlsl will be a reversed list of tokens.
+        (while (string-match "\\(\\S-+\\|\\s-+\\)" dls pos)
+          (push (match-string 1 dls) dlsl)
+          (setq pos (match-end 1)))
+
+        ;; Prepend trailing entries until they form an existing directory,
+        ;; whitespace and all.  Discard the next whitespace and repeat.
+        (while dlsl
+          (let ((newelt "")
+                tem1 tem2)
+            (while newelt
+              ;; We need tem1 because we don't want to prepend
+              ;; `comint-file-name-prefix' repeatedly into newelt via tem2.
+              (setq tem1 (pop dlsl)
+                    tem2 (concat comint-file-name-prefix tem1 newelt))
+              (cond ((file-directory-p tem2)
+                     (push tem2 ds)
+                     (when (string= " " (car dlsl))
+                       (pop dlsl))
+                     (setq newelt nil))
+                    (t
+                     (setq newelt (concat tem1 newelt)))))))
+
+        (with-demoted-errors "Couldn't cd: %s"
+          (shell-cd (car ds))
+          (setq shell-dirstack (cdr ds)
+                shell-last-dir (car shell-dirstack))
+          (shell-dirstack-message))))
     (if started-at-pmark (goto-char (marker-position pmark)))))
 
 ;; For your typing convenience:

bug#54776: 28.1; shell-resync-dirs hangs

[Kevin K. Lin]

Hi Lars,

I just tried the patch, and it works fine on my end.
Thanks!

Kevin

bug#54776: 28.1; shell-resync-dirs hangs

[Lars Ingebrigtsen]

"Kevin K. Lin" <klin@math.arizona.edu> writes:

> I just tried the patch, and it works fine on my end.

Thanks for testing; I've now reverted the offending change on the
emacs-28 branch, so it'll be included in the Emacs 28.2 release.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no