bug#34489: 25.2; pdb fails if directory contains '++'

bug#34489: 25.2; pdb fails if directory contains '++'

[a.soroa]

'pdb' command does not work if the directory where the python script is
contains the characters '++'. Here is a receipt

$ mkdir /tmp/foo++
$ cd /tmp/foo++
$ echo "print(10)" > pp.py
$ emacs -Q
M-x pdb
Run pdb (like this): python3 -m pdb pp.py




In GNU Emacs 25.2.2 (x86_64-pc-linux-gnu, GTK+ Version 3.22.21)
 of 2017-09-22, modified by Debian built on lgw01-amd64-050
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description:	Ubuntu 18.04.2 LTS

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --build x86_64-linux-gnu
 --prefix=/usr --sharedstatedir=/var/lib --libexecdir=/usr/lib
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-pop=yes
 --enable-locallisppath=/etc/emacs25:/etc/emacs:/usr/local/share/emacs/25.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/25.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-x=yes --with-x-toolkit=gtk3
 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs25-jYekUr/emacs25-25.2+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS NOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_MONETARY: en_IE.UTF-8
  value of $LC_NUMERIC: en_IE.UTF-8
  value of $LC_TIME: en_IE.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell

Minor modes in effect:
  shell-dirtrack-mode: t
  global-flycheck-mode: t
  diff-auto-refine-mode: t
  global-company-mode: t
  company-mode: t
  projectile-mode: t
  ido-vertical-mode: t
  save-place-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
libreoffice 2019-1-9-DK-Sailak-IIG-Bilera-\(Pepe\).odt: finished.
~ 
/tmp 
/tmp/k++ 
(New file)
Can’t guess python-indent-offset, using defaults: 4
Trailing newlines [trailing-newlines]
next-line: End of buffer
Saving file /tmp/k++/pp.py...
Wrote /tmp/k++/pp.py
Quit

Load-path shadows:
/usr/share/emacs/25.2/site-lisp/debian-startup hides /usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs/site-lisp/rst hides /usr/share/emacs/25.2/lisp/textmodes/rst
/home/ccpsoeta/.emacs.d/elpa/let-alist-1.0.5/let-alist hides /usr/share/emacs/25.2/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug gud pcmpl-unix conf-mode network-stream
nsm starttls url-cache url-http tls gnutls url-gw url-auth warnings
vc-git company-jedi jedi-core python-environment epc ctable concurrent
python tramp-sh misearch multi-isearch cal-move parse-time mm-archive
shr-color color shr dom browse-url notmuch hl-line notmuch-message
notmuch-hello notmuch-tree notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-draft notmuch-maildir-fcc notmuch-address
notmuch-company notmuch-parser notmuch-wash coolj notmuch-query
goto-addr icalendar diary-lib diary-loaddefs notmuch-tag crm notmuch-lib
notmuch-compat cl mm-view mml-smime smime dig org-agenda flyspell rect
tramp-cache tramp tramp-compat tramp-loaddefs trampver ucs-normalize
shell org-element org-rmail org-mhe org-irc org-info org-gnus
org-docview doc-view jka-compr image-mode org-bibtex bibtex org-bbdb
org-w3m org org-macro org-footnote org-pcomplete pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys ob-comint
ob-core ob-eval org-compat org-macs org-loaddefs cal-menu calendar
cal-loaddefs windmove dired-aux message sendmail format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader ispell flycheck find-func
company-ycmd f ycmd diff-mode diff request-deferred request mail-utils
url url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util url-parse auth-source gnus-util mm-util help-fns
mail-prsvr password-cache url-vars mailcap json map hmac-def deferred s
dash cus-edit wid-edit company-oddmuse company-keywords company-etags
etags xref project eieio eieio-core company-gtags company-dabbrev-code
company-dabbrev company-files company-capf company-cmake company-xcode
company-clang company-semantic company-eclim company-template
company-css company-nxml company-bbdb company projectile subr-x grep
compile comint ansi-color ring ibuf-ext ibuffer thingatpt flx-ido advice
flx ido-vertical-mode ido cus-start cus-load dired-x dired printing
ps-print ps-def lpr server pinentry epa derived epg saveplace cl-macs
cl-seq use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core edmacro kmacro finder-inf tex-site rx info package
epg-config seq byte-opt gv bytecomp byte-compile cl-extra help-mode
easymenu cconv cl-loaddefs pcase cl-lib time-date mule-util tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 684834 57692)
 (symbols 48 49680 0)
 (miscs 40 2836 1964)
 (strings 32 162971 25757)
 (string-bytes 1 4632525)
 (vectors 16 77915)
 (vector-slots 8 1988582 65164)
 (floats 8 910 634)
 (intervals 56 11276 252)
 (buffers 976 73))

bug#34489: 25.2; pdb fails if directory contains '++'

[Stefan Kangas]

a.soroa@ehu.eus (a.soroa) writes:

> 'pdb' command does not work if the directory where the python script is
> contains the characters '++'. Here is a receipt
>
> $ mkdir /tmp/foo++
> $ cd /tmp/foo++
> $ echo "print(10)" > pp.py
> $ emacs -Q
> M-x pdb
> Run pdb (like this): python3 -m pdb pp.py

I am able to reproduce this on current master.

The expected behaviour is to see TWO buffers in separate windows,
first "*gud-pp.py" with the following content:

    Current directory is /tmp/foo++/
    > /tmp/foo++/pp.py(1)<module>()
    -> print(10)
    (Pdb)

And the second a buffer visiting "pp.py".

The symptom is that you only see ONE buffer "*gud-pp.py*" with the
following content:

    Current directory is /tmp/foo++/

The bug (half) goes away as soon as I try to debug `gud-common-init'
using M-x edebug-defun, but only when stepping through manually using
"n" (and not when hitting "c").  Without this instrumentation, I get
the bug as above.  With it, I see the expected output in the
"*gud-pp.py*" buffer (but *not* the expected "pp.py" buffer).

I tried adding strategic calls to sit-for in gud-common-init and could
get the same result as with M-x edebug-defun.

Does anyone have any ideas what could cause this?

Thanks,
Stefan Kangas

bug#34489: 25.2; pdb fails if directory contains '++'

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Sun, 30 Jun 2019 18:13:19 +0200
> Cc: 34489@debbugs.gnu.org
> 
> The expected behaviour is to see TWO buffers in separate windows,
> first "*gud-pp.py" with the following content:
> 
>     Current directory is /tmp/foo++/
>     > /tmp/foo++/pp.py(1)<module>()
>     -> print(10)
>     (Pdb)
> 
> And the second a buffer visiting "pp.py".
> 
> The symptom is that you only see ONE buffer "*gud-pp.py*" with the
> following content:
> 
>     Current directory is /tmp/foo++/
> 
> The bug (half) goes away as soon as I try to debug `gud-common-init'
> using M-x edebug-defun, but only when stepping through manually using
> "n" (and not when hitting "c").  Without this instrumentation, I get
> the bug as above.  With it, I see the expected output in the
> "*gud-pp.py*" buffer (but *not* the expected "pp.py" buffer).
> 
> I tried adding strategic calls to sit-for in gud-common-init and could
> get the same result as with M-x edebug-defun.
> 
> Does anyone have any ideas what could cause this?

A stab in the dark: gud-pdb-marker-regexp?

bug#34489: 25.2; pdb fails if directory contains '++'

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> > Does anyone have any ideas what could cause this?
>
> A stab in the dark: gud-pdb-marker-regexp?

Indeed, that was the culprit.  I've attached a fix.

Thanks,
Stefan Kangas

[-- Attachment #2: 0001-Fix-pdb-when-absolute-file-name-contains.patch --]
[-- Type: application/x-patch, Size: 1109 bytes --]

bug#34489: 25.2; pdb fails if directory contains '++'

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Sun, 30 Jun 2019 19:24:56 +0200
> Cc: a.soroa@ehu.eus, 34489@debbugs.gnu.org
> 
> > A stab in the dark: gud-pdb-marker-regexp?
> 
> Indeed, that was the culprit.  I've attached a fix.

Hmm... I wonder why the code bothers specifying the allowed characters
explicitly, and in particular why it only allows ASCII characters.
Does the code work if the directory has non-ASCII characters instead
of "++"?

Thanks.

bug#34489: 25.2; pdb fails if directory contains '++'

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 554 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> Hmm... I wonder why the code bothers specifying the allowed characters
> explicitly, and in particular why it only allows ASCII characters.
>
> Does the code work if the directory has non-ASCII characters instead
> of "++"?

No, it breaks.  For now, I've attached a patch to use "[:alnum:]",
which fixes that use case for me.

Regarding your first question, I'm not exactly sure why.  Maybe we
could go as far as just doing "[[:print:]]*" for that part - or
simply ".*".  What do you think?

Thanks,
Stefan Kangas

[-- Attachment #2: 0001-Make-M-x-pdb-handle-more-valid-file-names.patch --]
[-- Type: text/x-patch, Size: 1105 bytes --]

From 2f671d7e84109af649457cf077ca10e949c5211d Mon Sep 17 00:00:00 2001
From: Stefan Kangas <stefankangas@gmail.com>
Date: Sun, 30 Jun 2019 19:17:52 +0200
Subject: [PATCH] Make "M-x pdb" handle more valid file names

* lisp/progmodes/gud.el (gud-pdb-marker-regexp): Add "+" and
"[:alnum:]" to the regexp.  (Bug#34489)
---
 lisp/progmodes/gud.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/progmodes/gud.el b/lisp/progmodes/gud.el
index 4306f5daa0..fdf53ff2e7 100644
--- a/lisp/progmodes/gud.el
+++ b/lisp/progmodes/gud.el
@@ -1606,7 +1606,7 @@ gud-pdb-history
 ;; Last group is for return value, e.g. "> test.py(2)foo()->None"
 ;; Either file or function name may be omitted: "> <string>(0)?()"
 (defvar gud-pdb-marker-regexp
-  "^> \\([-a-zA-Z0-9_/.:@ \\]*\\|<string>\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|<module>\\)()\\(->[^\n\r]*\\)?[\n\r]")
+  "^> \\([-[:alnum:]_+/.:@ \\]*\\|<string>\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|<module>\\)()\\(->[^\n\r]*\\)?[\n\r]")
 
 (defvar gud-pdb-marker-regexp-file-group 1)
 (defvar gud-pdb-marker-regexp-line-group 2)
-- 
2.11.0

bug#34489: 25.2; pdb fails if directory contains '++'

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Tue, 2 Jul 2019 21:42:31 +0200
> Cc: a.soroa@ehu.eus, 34489@debbugs.gnu.org
> 
> 
> [1:text/plain Hide]
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Hmm... I wonder why the code bothers specifying the allowed characters
> > explicitly, and in particular why it only allows ASCII characters.
> >
> > Does the code work if the directory has non-ASCII characters instead
> > of "++"?
> 
> No, it breaks.  For now, I've attached a patch to use "[:alnum:]",
> which fixes that use case for me.
> 
> Regarding your first question, I'm not exactly sure why.  Maybe we
> could go as far as just doing "[[:print:]]*" for that part - or
> simply ".*".  What do you think?

I think both [:print:] and .* would be too radical, as I'm not sure
including control characters and arbitrary whitespace will not break
something.  But maybe [:graph:] is better than [:alnum:].

In any case, I think we want a comment there saying that this is to
allow more characters in file names shown in the prompt.

Thanks.

bug#34489: 25.2; pdb fails if directory contains '++'

[Stefan Kangas]

[-- Attachment #1: Type: text/plain, Size: 424 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> I think both [:print:] and .* would be too radical, as I'm not sure
> including control characters and arbitrary whitespace will not break
> something.  But maybe [:graph:] is better than [:alnum:].
>
> In any case, I think we want a comment there saying that this is to
> allow more characters in file names shown in the prompt.

How about the attached patch?

Thanks,
Stefan Kangas

[-- Attachment #2: 0001-Make-M-x-pdb-use-graph-to-match-file-names.patch --]
[-- Type: text/x-patch, Size: 1293 bytes --]

From 1e78d0abd4bc1d9475265c3b87718266fe5e9c92 Mon Sep 17 00:00:00 2001
From: Stefan Kangas <stefankangas@gmail.com>
Date: Sun, 30 Jun 2019 19:17:52 +0200
Subject: [PATCH] Make "M-x pdb" use "[:graph:]" to match file names

* lisp/progmodes/gud.el (gud-pdb-marker-regexp): Use "[:graph:]" to
match file name in prompt.  (Bug#34489)
---
 lisp/progmodes/gud.el | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/lisp/progmodes/gud.el b/lisp/progmodes/gud.el
index 4306f5daa0..6b152b7b90 100644
--- a/lisp/progmodes/gud.el
+++ b/lisp/progmodes/gud.el
@@ -1605,8 +1605,12 @@ gud-pdb-history
 
 ;; Last group is for return value, e.g. "> test.py(2)foo()->None"
 ;; Either file or function name may be omitted: "> <string>(0)?()"
+;;
+;; We use [:graph:] to be very allowing with regards to which
+;; characters we match in the file name shown in the prompt.
+;; (Of course, this matches the "<string>" case too.)
 (defvar gud-pdb-marker-regexp
-  "^> \\([-a-zA-Z0-9_/.:@ \\]*\\|<string>\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|<module>\\)()\\(->[^\n\r]*\\)?[\n\r]")
+  "^> \\([[:graph:] \\]*\\)(\\([0-9]+\\))\\([a-zA-Z0-9_]*\\|\\?\\|<module>\\)()\\(->[^\n\r]*\\)?[\n\r]")
 
 (defvar gud-pdb-marker-regexp-file-group 1)
 (defvar gud-pdb-marker-regexp-line-group 2)
-- 
2.11.0

bug#34489: 25.2; pdb fails if directory contains '++'

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Date: Fri, 5 Jul 2019 17:51:00 +0200
> Cc: a.soroa@ehu.eus, 34489@debbugs.gnu.org
> 
> > I think both [:print:] and .* would be too radical, as I'm not sure
> > including control characters and arbitrary whitespace will not break
> > something.  But maybe [:graph:] is better than [:alnum:].
> >
> > In any case, I think we want a comment there saying that this is to
> > allow more characters in file names shown in the prompt.
> 
> How about the attached patch?

Thanks, pushed to the master branch.