bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Robin Green]

This bug report will be sent to the Free Software Foundation,
not to your local site managers!
Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list,
and to the gnu.emacs.bug news group.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug.  If you can, give
a recipe starting from `emacs -Q':

I did M-x rgrep and searched for a string which happened to be matched by some files, and one of the filenames happened to contain a colon character (':'). Emacs thought that the colon character indicated a line number would immediately follow, and it syntax highlighted the buffer accordingly. When I tried to visit the file in question by navigating to that line and pressing Enter, emacs asked me for which file to visit, because it had truncated the filename at the colon character.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/share/emacs/23.2/etc/DEBUG.


In GNU Emacs 23.2.1 (i686-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2010-05-22 on cspcnh.swan.ac.uk
Windowing system distributor `The X.Org Foundation', version 11.0.10900000
configured using `configure  '--prefix=/usr' '--host=i686-pc-linux-gnu' '--build=i686-pc-linux-gnu' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--datadir=/usr/share' '--docdir=/usr/share/doc/emacs-23.2' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--disable-dependency-tracking' '--disable-silent-rules' '--enable-fast-install' '--libdir=/usr/lib' '--infodir=/usr/share/info/emacs' '--without-kerberos' '--without-kerberos5' '--without-hesiod' '--without-m17n-flt' '--without-xim' '--with-x' '--without-sound' '--with-xft' '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars' '--without-libotf' '--with-rsvg' '--without-dbus' '--with-gif' '--without-gpm' '--with-jpeg' '--with-png' '--without-tiff' '--with-xpm' '--without-gconf' 'build_alias=i686-pc-linux-gnu' 'host_alias=i686-pc-linux
 -gnu' 'CC=i686-pc-linux-gnu-gcc' 'CFLAGS=-march=native -pipe -O2' 'CPP=cpp''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Grep

Minor modes in effect:
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
i t ? <help-echo> <backspace> <backspace> <backspace> 
t h a t SPC f i l e ? SPC h w y ? <backspace> <backspace> 
<backspace> <backspace> w h y ? <return> <help-echo> 
<switch-frame> s <switch-frame> <help-echo> S o SPC 
w h a t SPC i s SPC t h e SPC " r i g h t SPC w a y 
" SPC t o SPC m a k e SPC a SPC w r a p p e r SPC s 
c r i p t ? S-SPC J u s t SPC u s e SPC h e r e b i 
n ? <return> <help-echo> C-x b <tab> n c - 4 <tab> 
<backspace> <backspace> <backspace> x - <tab> 4 <tab> 
<return> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> M-x 
r g r e p <return> h e r e b i n <return> * . e x * 
<return> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <return> M-x r g r e p <return> 
h e r e b i n <return> * . e x * <return> <home> C-k 
/ v a r / d b / p a l <tab> r e <tab> <return> <M-right> 
<M-left> C-x C-f <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
/ s c <tab> c <tab> C-g <M-right> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <return> C-g <help-echo> <switch-frame> 
<switch-frame> M-x r e p o r t - e m <tab> <return
>

Recent messages:
Checking "%inbox" is done
Checking "%inbox"
Checking "%inbox" is done
Checking "%inbox"
Checking "%inbox" is done
Making completion list... [2 times]
Grep exited abnormally with code 123
Grep finished (matches found)
Quit
Mark set
Quit

Load-path shadows:
/usr/share/emacs/site-lisp/coq hides /usr/share/emacs/site-lisp/ProofGeneral/coq/coq
/usr/share/emacs/site-lisp/coq-syntax hides /usr/share/emacs/site-lisp/ProofGeneral/coq/coq-syntax
/usr/share/emacs/site-lisp/coq-db hides /usr/share/emacs/site-lisp/ProofGeneral/coq/coq-db
/usr/share/emacs/site-lisp/semi/pgg-gpg hides /usr/share/emacs/23.2/lisp/pgg-gpg
/usr/share/emacs/site-lisp/flim/hex-util hides /usr/share/emacs/23.2/lisp/hex-util
/usr/share/emacs/site-lisp/semi/pgg-parse hides /usr/share/emacs/23.2/lisp/pgg-parse
/usr/share/emacs/site-lisp/semi/pgg-pgp5 hides /usr/share/emacs/23.2/lisp/pgg-pgp5
/usr/share/emacs/site-lisp/flim/md4 hides /usr/share/emacs/23.2/lisp/md4
/usr/share/emacs/site-lisp/flim/sha1 hides /usr/share/emacs/23.2/lisp/sha1
/usr/share/emacs/site-lisp/semi/pgg hides /usr/share/emacs/23.2/lisp/pgg
/usr/share/emacs/site-lisp/semi/pgg-def hides /usr/share/emacs/23.2/lisp/pgg-def
/usr/share/emacs/site-lisp/semi/pgg-pgp hides /usr/share/emacs/23.2/lisp/pgg-pgp
/usr/share/emacs/site-lisp/flim/hmac-def hides /usr/share/emacs/23.2/lisp/net/hmac-def
/usr/share/emacs/site-lisp/flim/sasl-ntlm hides /usr/share/emacs/23.2/lisp/net/sasl-ntlm
/usr/share/emacs/site-lisp/flim/hmac-md5 hides /usr/share/emacs/23.2/lisp/net/hmac-md5
/usr/share/emacs/site-lisp/flim/sasl-cram hides /usr/share/emacs/23.2/lisp/net/sasl-cram
/usr/share/emacs/site-lisp/flim/sasl hides /usr/share/emacs/23.2/lisp/net/sasl
/usr/share/emacs/site-lisp/flim/ntlm hides /usr/share/emacs/23.2/lisp/net/ntlm
/usr/share/emacs/site-lisp/flim/sasl-digest hides /usr/share/emacs/23.2/lisp/net/sasl-digest
/usr/share/emacs/site-lisp/wl/rfc2368 hides /usr/share/emacs/23.2/lisp/mail/rfc2368
/usr/share/emacs/site-lisp/wl/utf7 hides /usr/share/emacs/23.2/lisp/gnus/utf7
/usr/share/emacs/site-lisp/semi/smime hides /usr/share/emacs/23.2/lisp/gnus/smime

Features:
(shadow emacsbug erc-menu erc-join erc-ring erc-networks erc-pcomplete
erc-track erc-match erc-button erc-fill erc-stamp erc-netsplit
erc-goodies erc erc-backend erc-compat format-spec find-dired grep
compile help-mode ansi-color shell comint smtp sasl sasl-anonymous
sasl-login sasl-plain pcomplete vc-git info cus-edit wid-edit
multi-isearch tabify man view dired-x dired-aux dired wl-score
elmo-internal mule-util mel-q-ccl elmo-nntp elmo-imap4 time-stamp
elmo-net elmo-dop elmo-cache elmo-map modb-standard wl-mime mime-edit
pgg-parse pccl pccl-20 ccl signature mail-utils mime-setup
mail-mime-setup semi-setup mime-pgp pgg-def mime-play filename
mime-image wl-demo wl-draft eword-encode wl-template sendmail wl-news
derived wl-address wl-thread wl-action wl-summary ps-print ps-def lpr
wl-refile wl-message elmo-mime mmelmo-buffer mmelmo-imap mime-view
mime-conf calist semi-def mmimap mime-parse mmbuffer mmgeneric
wl-highlight elmo-multi wl-folder wl wl-e21 wl-util pp elmo-flag
elmo-localdir wl-vars epg-config wl-version elmo elmo-signal
elmo-msgdb modb modb-generic modb-entity mime elmo-util emu invisible
inv-19 poem poem-e20 poem-e20_3 utf7 eword-decode mel mime-def alist
std11 mcharset mcs-20 mcs-e20 pces pces-e20 pces-20 broken pcustom
elmo-date path-util elmo-vars poe elmo-version pym static apel-ver
product luna avoid magit-topgit cl cl-19 magit-svn magit diff-mode
log-edit pcvs-util add-log magit-key-mode assoc windmove emacsd-tile
edmacro kmacro winner ring server remember org-remember org-datetree
org byte-opt bytecomp byte-compile advice help-fns advice-preload
org-footnote org-src org-list org-faces org-compat org-macs easymenu
time-date noutline outline easy-mmode paren cus-start cus-load
site-exherbo eclectic-mode exheres-mode exheres-mode-keywords skeleton
sh-script executable cmake-mode thingatpt bbdb-autoloads bbdb
regexp-opt timezone proof-site proof-autoloads pg-vars tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd font-setting
tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mldrag mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button
minibuffer faces cus-face files text-properties overlay md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process font-render-setting gtk x-toolkit x
multi-tty emacs)

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Stefan Monnier]

> I did M-x rgrep and searched for a string which happened to be matched
> by some files, and one of the filenames happened to contain a colon
> character (':'). Emacs thought that the colon character indicated
> a line number would immediately follow, and it syntax highlighted the
> buffer accordingly. When I tried to visit the file in question by
> navigating to that line and pressing Enter, emacs asked me for which
> file to visit, because it had truncated the filename at the
> colon character.

AFAIK we do try to figure out what was meant, even though in general
grep's output is simply ambiguous when the file name has colons.

So, could you give the exact file name you used, so we can try and
figure out whether it's a bug in the current heuristic, or it's
a hopeless case (it's just too ambiguous), or it's a case that's simply
not yet handled.


        Stefan

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Robin Green]

At Thu, 11 Nov 2010 18:09:38 -0500,
Stefan Monnier wrote:
> AFAIK we do try to figure out what was meant, even though in general
> grep's output is simply ambiguous when the file name has colons.
> 
> So, could you give the exact file name you used

./installed/data/x11-libs---nx/3.4.0:0:C.30253.1289557929.792611.C/nx-3.4.0.exheres-0

Thanks

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Stefan Monnier]

>> AFAIK we do try to figure out what was meant, even though in general
>> grep's output is simply ambiguous when the file name has colons.
>> So, could you give the exact file name you used
> ./installed/data/x11-libs---nx/3.4.0:0:C.30253.1289557929.792611.C/nx-3.4.0.exheres-0

I guess strictly speaking it's not ambiguous since grep normally starts
counting from 1, so the ":0:" cannot actually be the line number
(e.g. the patch below could help).
But I also expect that among your file names, the ":0:" is just an
accident and it could just as well be ":5:" instead, in which case
greps' output will be *really* ambiguous.

I.e. to resolve this kind of ambiguity, grep-mode would have to check
all the possible interpretations and see which ones lead to an existing
file name.  Such a change would require a good bit more work on grep.
Furthermore such ambiguity would need to be resolved not only when
the ":N:" comes from the file name but also when it comes from the
file's content.


        Stefan


=== modified file 'lisp/progmodes/grep.el'
--- lisp/progmodes/grep.el	2010-05-21 20:43:04 +0000
+++ lisp/progmodes/grep.el	2010-11-15 16:28:07 +0000
@@ -348,7 +348,11 @@
     ;; produces them
     ;; ("^\\(.+?\\)\\(:[ \t]*\\)\\([0-9]+\\)\\2\\(?:\\([0-9]+\\)\\(?:-\\([0-9]+\\)\\)?\\2\\)?"
     ;;  1 3 (4 . 5))
-    ("^\\(\\(.+?\\):\\([0-9]+\\):\\).*?\
+    ;; Note that we want to use as tight a regexp as we can to try and
+    ;; handle weird file names (with colons in them) as well as possible.
+    ;; E.g. we use [1-9][0-9]* rather than [0-9]+ so as to accept ":034:" in
+    ;; file names.
+    ("^\\(\\(.+?\\):\\([1-9][0-9]*\\):\\).*?\
 \\(\033\\[01;31m\\(?:\033\\[K\\)?\\)\\(.*?\\)\\(\033\\[[0-9]*m\\)"
      2 3
      ;; Calculate column positions (beg . end) of first grep match on a line

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Juri Linkov]

> I.e. to resolve this kind of ambiguity, grep-mode would have to check
> all the possible interpretations and see which ones lead to an existing
> file name.  Such a change would require a good bit more work on grep.
> Furthermore such ambiguity would need to be resolved not only when
> the ":N:" comes from the file name but also when it comes from the
> file's content.

One more case where highlighting is currently incorrect is grepping mail
archives where each mail is located in a separate file with the timestamp
in its name like:

grep -inH -e "Date: " *
2011-08-31_11:57:03_1:7:Date: Wed, 31 Aug 2011 11:57:03 +0000
2011-08-31_12:08:20_2:7:Date: Wed, 31 Aug 2011 12:08:20 +0000

In the first line "2011-08-31_11" is highlighted as the file name and
"57" as the line number.  In the second line "2011-08-31_12:08:20_2" is
highlighted as the file name (correctly) and "7" as the line number
(the difference is because "03" has the leading zero).

I see no way to match file names and line numbers correctly
with a regexp in such cases.

The only way to do this reliably with GNU grep is to ask it to output
escape sequences around file names and line numbers.  From `man grep':

  fn=35 SGR substring for file names prefixing any content line.
  The default is a magenta text foreground over the terminal's
  default background.

  ln=32 SGR substring for line numbers prefixing any content line.
  The default is a green text foreground over the terminal's
  default background.

Then `grep-filter' could find these escape sequences and highlight,
and `grep-regexp-alist' could calculate column positions of the highlighted
file names and line numbers.

Is it worth a try?

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[Stefan Monnier]

> The only way to do this reliably with GNU grep is to ask it to output
> escape sequences around file names and line numbers.  From `man grep':

>   fn=35 SGR substring for file names prefixing any content line.
>   The default is a magenta text foreground over the terminal's
>   default background.

>   ln=32 SGR substring for line numbers prefixing any content line.
>   The default is a green text foreground over the terminal's
>   default background.

> Then `grep-filter' could find these escape sequences and highlight,
> and `grep-regexp-alist' could calculate column positions of the highlighted
> file names and line numbers.

> Is it worth a try?

Feel free to try, but not for 24.1.  And please try and quantify the
performance impact (if any), because M-x grep is already too slow.


        Stefan

bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

tags 7378 + patch
quit

>> The only way to do this reliably with GNU grep is to ask it to output
>> escape sequences around file names and line numbers.

> Feel free to try, but not for 24.1.  And please try and quantify the
> performance impact (if any), because M-x grep is already too slow.

GNU grep also supports --null to output NUL bytes following file names,
I haven't measured but it should be no slower and possibly faster since
the regexp is simpler.  There is a bit of added complexity to still
support other greps which don't have --null:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 10953 bytes --]

From 0042aa5616560657e867aa977dad732ab18e30dc Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Tue, 13 Sep 2016 20:48:09 -0400
Subject: [PATCH] Use grep's --null option (Bug#6843)

* lisp/progmodes/grep.el (grep-use-null-filename-separator): New option.
(grep--regexp-alist-column, grep--regexp-alist-bin-matcher)
(grep-with-null-regexp-alist, grep-fallback-regexp-alist): New
constants, replacing `grep-regexp-alist'.
(grep-regex-alist): Mark the variable obsolete, add a new function of
the same name to replace it.
(grep-compute-defaults): Compute default for
`grep-use-null-filename-separator'.
(grep-mode): Set compilation-error-regexp-alist (buffer locally) to the
value of `grep-with-null-regexp-alist' or `grep-fallback-regexp-alist'
according to `grep-use-null-filename-separator'.
* lisp/progmodes/xref.el (xref-collect-matches): Call
`grep-regex-alist' instead of the obsolete variable.  Don't hardcode
grep-regexp-alist match groups.
---
 lisp/progmodes/grep.el | 103 +++++++++++++++++++++++++++++++++++--------------
 lisp/progmodes/xref.el |  33 ++++++++--------
 2 files changed, 92 insertions(+), 44 deletions(-)

diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index b3d8a51cee..a4b85bbbe4 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -47,8 +47,8 @@ (defvar grep-host-defaults-alist nil
 (defun grep-apply-setting (symbol value)
   "Set SYMBOL to VALUE, and update `grep-host-defaults-alist'.
 SYMBOL should be one of `grep-command', `grep-template',
-`grep-use-null-device', `grep-find-command',
-`grep-find-template', `grep-find-use-xargs', or
+`grep-use-null-device', `grep-find-command' `grep-find-template',
+`grep-find-use-xargs', `grep-use-null-filename-separator', or
 `grep-highlight-matches'."
   (when grep-host-defaults-alist
     (let* ((host-id
@@ -160,6 +160,15 @@ (defcustom grep-use-null-device 'auto-detect
   :set 'grep-apply-setting
   :group 'grep)
 
+(defcustom grep-use-null-filename-separator 'auto-detect
+  "If non-nil, use `grep's `--null' option.
+This is done to disambiguate file names in `grep's output."
+  :type '(choice (const :tag "Do Not Use `--null'" nil)
+		 (const :tag "Use `--null'" t)
+		 (other :tag "Not Set" auto-detect))
+  :set 'grep-apply-setting
+  :group 'grep)
+
 ;;;###autoload
 (defcustom grep-find-command nil
   "The default find command for \\[grep-find].
@@ -357,33 +366,53 @@ (defvar grep-last-buffer nil
 Notice that using \\[next-error] or \\[compile-goto-error] modifies
 `compilation-last-buffer' rather than `grep-last-buffer'.")
 
-;;;###autoload
-(defconst grep-regexp-alist
-  '(
-    ;; Use a tight regexp to handle weird file names (with colons
+(defconst grep--regexp-alist-column
+  ;; Calculate column positions (col . end-col) of first grep match on a line
+  (cons
+   (lambda ()
+     (when grep-highlight-matches
+       (let* ((beg (match-end 0))
+              (end (save-excursion (goto-char beg) (line-end-position)))
+              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face)))
+         (when mbeg
+           (- mbeg beg)))))
+   (lambda ()
+     (when grep-highlight-matches
+       (let* ((beg (match-end 0))
+              (end (save-excursion (goto-char beg) (line-end-position)))
+              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face))
+              (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
+         (when mend
+           (- mend beg)))))))
+(defconst grep--regexp-alist-bin-matcher
+  '("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
+(defconst grep-with-null-regexp-alist
+  `(("^\\([^\0]+\\)\\(\0\\)\\([0-9]+\\):" 1 3 ,grep--regexp-alist-column nil nil
+     (2 '(face unspecified display ":")))
+    ,grep--regexp-alist-bin-matcher)
+  "Regexp used to match grep hits.
+See `compilation-error-regexp-alist'.")
+(defconst grep-fallback-regexp-alist
+  `(;; Use a tight regexp to handle weird file names (with colons
     ;; in them) as well as possible.  E.g., use [1-9][0-9]* rather
     ;; than [0-9]+ so as to accept ":034:" in file names.
     ("^\\(.*?[^/\n]\\):[ \t]*\\([1-9][0-9]*\\)[ \t]*:"
-     1 2
-     ;; Calculate column positions (col . end-col) of first grep match on a line
-     ((lambda ()
-	(when grep-highlight-matches
-	  (let* ((beg (match-end 0))
-		 (end (save-excursion (goto-char beg) (line-end-position)))
-		 (mbeg (text-property-any beg end 'font-lock-face grep-match-face)))
-	    (when mbeg
-	      (- mbeg beg)))))
-      .
-      (lambda ()
-	(when grep-highlight-matches
-	  (let* ((beg (match-end 0))
-		 (end (save-excursion (goto-char beg) (line-end-position)))
-		 (mbeg (text-property-any beg end 'font-lock-face grep-match-face))
-		 (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
-	    (when mend
-	      (- mend beg)))))))
-    ("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
-  "Regexp used to match grep hits.  See `compilation-error-regexp-alist'.")
+     1 2 ,grep--regexp-alist-column)
+    ,grep--regexp-alist-bin-matcher)
+  "Regexp used to match grep hits when `--null' is not supported.
+See `compilation-error-regexp-alist'.")
+
+(defvaralias 'grep-regex-alist 'grep-with-null-regexp-alist)
+(make-obsolete-variable
+ 'grep-regex-alist "Call `grep-regexp-alist' instead." "26.1")
+
+;;;###autoload
+(defun grep-regexp-alist ()
+  "Return a regexp alist to match grep hits.
+The regexp used depends on `grep-use-null-filename-separator'.
+See `compilation-error-regexp-alist' for format details."
+  (if grep-use-null-filename-separator
+      grep-with-null-regexp-alist grep-fallback-regexp-alist))
 
 (defvar grep-first-column 0		; bug#10594
   "Value to use for `compilation-first-column' in grep buffers.")
@@ -538,6 +567,7 @@ (defun grep-compute-defaults ()
 	     (grep-use-null-device ,grep-use-null-device)
 	     (grep-find-command ,grep-find-command)
 	     (grep-find-template ,grep-find-template)
+             (grep-use-null-filename-separator ,grep-use-null-filename-separator)
 	     (grep-find-use-xargs ,grep-find-use-xargs)
 	     (grep-highlight-matches ,grep-highlight-matches)))))
   (let* ((host-id
@@ -550,7 +580,8 @@ (defun grep-compute-defaults ()
     ;; computed for every host once.
     (dolist (setting '(grep-command grep-template
 		       grep-use-null-device grep-find-command
-		       grep-find-template grep-find-use-xargs
+		       grep-use-null-filename-separator
+                       grep-find-template grep-find-use-xargs
 		       grep-highlight-matches))
       (set setting
 	   (cadr (or (assq setting host-defaults)
@@ -576,6 +607,21 @@ (defun grep-compute-defaults ()
 			 (concat (regexp-quote hello-file)
 				 ":[0-9]+:English")))))))))
 
+    (when (eq grep-use-null-filename-separator 'auto-detect)
+      (setq grep-use-null-filename-separator
+            (with-temp-buffer
+              (let* ((hello-file (expand-file-name "HELLO" data-directory))
+                     (args `("--null" "-ne" "^English" ,hello-file)))
+                (if grep-use-null-device
+                    (setq args (append args (list null-device)))
+                  (push "-H" args))
+                (and (grep-probe grep-program `(nil t nil ,@args))
+                     (progn
+                       (goto-char (point-min))
+                       (looking-at
+                        (concat (regexp-quote hello-file)
+                                "\0[0-9]+:English"))))))))
+
     (when (eq grep-highlight-matches 'auto-detect)
       (setq grep-highlight-matches
 	    (with-temp-buffer
@@ -591,6 +637,7 @@ (defun grep-compute-defaults ()
 		 grep-template grep-find-template)
       (let ((grep-options
 	     (concat (if grep-use-null-device "-n" "-nH")
+                     (if grep-use-null-filename-separator " --null")
 		     (if (grep-probe grep-program
 				     `(nil nil nil "-e" "foo" ,null-device)
 				     nil 1)
@@ -733,7 +780,7 @@ (define-compilation-mode grep-mode "Grep"
   (set (make-local-variable 'compilation-error-face)
        grep-hit-face)
   (set (make-local-variable 'compilation-error-regexp-alist)
-       grep-regexp-alist)
+       (grep-regexp-alist))
   ;; compilation-directory-matcher can't be nil, so we set it to a regexp that
   ;; can never match.
   (set (make-local-variable 'compilation-directory-matcher) '("\\`a\\`"))
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index b8ec50f14a..cc9b794c5a 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -917,20 +917,21 @@ (defun xref-collect-matches (regexp files dir ignores)
   (grep-compute-defaults)
   (defvar grep-find-template)
   (defvar grep-highlight-matches)
-  (let* ((grep-find-template (replace-regexp-in-string "<C>" "<C> -E"
-                                                       grep-find-template t t))
-         (grep-highlight-matches nil)
-         ;; TODO: Sanitize the regexp to remove Emacs-specific terms,
-         ;; so that Grep can search for the "relaxed" version.  Can we
-         ;; do that reliably enough, without creating false negatives?
-         (command (xref--rgrep-command (xref--regexp-to-extended regexp)
-                                       files
-                                       (expand-file-name dir)
-                                       ignores))
-         (buf (get-buffer-create " *xref-grep*"))
-         (grep-re (caar grep-regexp-alist))
-         status
-         hits)
+  (pcase-let*
+      ((grep-find-template (replace-regexp-in-string "<C>" "<C> -E"
+                                                     grep-find-template t t))
+       (grep-highlight-matches nil)
+       ;; TODO: Sanitize the regexp to remove Emacs-specific terms,
+       ;; so that Grep can search for the "relaxed" version.  Can we
+       ;; do that reliably enough, without creating false negatives?
+       (command (xref--rgrep-command (xref--regexp-to-extended regexp)
+                                     files
+                                     (expand-file-name dir)
+                                     ignores))
+       (buf (get-buffer-create " *xref-grep*"))
+       (`(,grep-re ,file-group ,line-group . ,_) (car (grep-regexp-alist)))
+       (status nil)
+       (hits nil))
     (with-current-buffer buf
       (erase-buffer)
       (setq status
@@ -944,8 +945,8 @@ (defun xref-collect-matches (regexp files dir ignores)
                  (not (looking-at grep-re)))
         (user-error "Search failed with status %d: %s" status (buffer-string)))
       (while (re-search-forward grep-re nil t)
-        (push (list (string-to-number (match-string 2))
-                    (match-string 1)
+        (push (list (string-to-number (match-string line-group))
+                    (match-string file-group)
                     (buffer-substring-no-properties (point) (line-end-position)))
               hits)))
     (xref--convert-hits (nreverse hits) regexp)))
-- 
2.11.1

bug#6843: bug#7378: 23.2; grep buffer misinterprets result if filename contains colon character

[npostavs]

tags 7378 fixed
close 7378 26.1
quit

npostavs@users.sourceforge.net writes:
>
> GNU grep also supports --null to output NUL bytes following file names,
> I haven't measured but it should be no slower and possibly faster since
> the regexp is simpler.

Pushed to master.

[1: 644cdd1aa0]: 2017-07-19 20:03:00 -0400
  Use grep's --null option (Bug#6843)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=644cdd1aa0a10dbfffa3b9b4c7a97f8cddded0b8