bug#27840: 26.0.50; vc-git-grep stopped working on git master

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[Tom Tromey]

I am running a build of git master from yesterday.
I tried M-x vc-git-grep.

This successfully runs the grep and puts the output into *grep*.
However, the highlighting is incorrect and M-x next-error claims there
are no grep hits.  Using C-c (compile-goto-error) in *grep* also does
not work.



In GNU Emacs 26.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version 3.22.17)
 of 2017-07-25 built on pokyo
Repository revision: 60b74190286afdc85ccf4e5042a5b89732e2a320
Windowing system distributor 'Fedora Project', version 11.0.11903000
System Description:	Fedora release 25 (Twenty Five)

Recent messages:
Wrote /home/tromey/.newsrc.eld
Saving /home/tromey/.newsrc.eld...done
nnimap read 0k from imap.gmail.com
Expiring articles...done
Saving Gnus registry (92421 entries) to ~/.gnus.registry.eieio...
Saving Gnus registry (size 92421) to ~/.gnus.registry.eieio...done
Saving /home/tromey/.newsrc.eld...
Saving file /home/tromey/.newsrc.eld...
Wrote /home/tromey/.newsrc.eld
Saving /home/tromey/.newsrc.eld...done

Configured using:
 'configure --prefix=/home/tromey/Emacs/install --with-modules'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GSETTINGS NOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS
GTK3 X11 MODULES LIBSYSTEMD

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  erc-spelling-mode: t
  erc-truncate-mode: t
  diff-auto-refine-mode: t
  gnus-topic-mode: t
  gnus-undo-mode: t
  which-function-mode: t
  projectile-global-mode: t
  projectile-mode: t
  erc-notify-mode: t
  erc-notifications-mode: t
  erc-services-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-netsplit-mode: t
  erc-hl-nicks-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  savehist-mode: t
  cl-old-struct-compat-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/tromey/.emacs.d/elpa/bubbles-0.5/bubbles hides /home/tromey/Emacs/install/share/emacs/26.0.50/lisp/play/bubbles

Features:
(shadow emacsbug dxr idutils gnus-html help-fns radix-tree url-cache
erc-spelling erc-truncate cursor-sensor mhtml-mode css-mode smie eww
mm-url url-queue bbdb-sc supercite regi mailalias mail-hist nnir
git-rebase-mode git-commit-mode log-edit term/xterm xterm dabbrev
copyright find-dired misearch multi-isearch smerge-mode js sgml-mode
cc-mode cc-fonts cc-guess cc-menus cc-cmds bug-reference add-log
log-view pcvs-util webjump flow-fill bbdb-gui bbdb-hooks bbdb-rmail
rmailsum vc-mtn vc-hg vc-git diff-mode vc-bzr vc-src vc-sccs vc-svn
vc-cvs vc-rcs sort gnus-cite smiley shr-color url-http url url-proxy
url-privacy url-history url-auth url-cookie url-domsuf url-gw url-expand
url-methods url-util shr svg dom browse-url mm-archive gnus-bcklg
gnus-async qp gnus-ml disp-table gnus-topic nndraft nnmh nnfolder utf-7
bbdb-gnus bbdb-snarf mail-extr bbdb-com network-stream nsm starttls
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg nntp
gnus-cache gnus-registry registry eieio-base gnus-art mm-uu mml2015
mm-view mml-smime smime dig mailcap gnus-sum gnus-group gnus-undo
smtpmail sendmail gnus-start gnus-cloud nnimap nnmail mail-source tls
gnutls utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range message
puny rfc822 mml mml-sec epa derived epg mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader gnus-win gnus
nnheader gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
mail-utils mm-util mail-prsvr flyspell ispell diminish appt diary-lib
diary-loaddefs cal-menu calendar cal-loaddefs which-func imenu edmacro
kmacro projectile grep compile ibuf-ext ibuffer ibuffer-loaddefs
autorevert filenotify desktop frameset cus-start cus-load erc-notify
erc-desktop-notifications erc-services notifications dbus xml erc-list
erc-menu erc-join erc-ring erc-networks erc-pcomplete pcomplete
erc-track erc-match erc-netsplit erc-hl-nicks easy-mmode color
erc-button erc-fill erc-stamp wid-edit erc-goodies erc erc-backend
erc-compat format-spec thingatpt pp dired-aux dired-x dired
dired-loaddefs warnings advice vc-dir ewoc vc vc-dispatcher flycheck
cl-extra json map find-func help-mode rx subr-x dash cc-styles cc-align
cc-engine cc-vars cc-defs bbdb cl timezone ange-ftp comint ansi-color
ring server savehist finder-inf bbdb-autoloads dwarf-mode-autoloads
lisppaste-autoloads pydoc-info-autoloads info-look weblogger-autoloads
info package easymenu epg-config url-handlers url-parse auth-source
cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache url-vars
seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 3013500 1806998)
 (symbols 48 106763 55)
 (miscs 40 24752 5430)
 (strings 32 1386985 121382)
 (string-bytes 1 63141799)
 (vectors 16 114896)
 (vector-slots 8 3399404 294794)
 (floats 8 535 2243)
 (intervals 56 16523 3246)
 (buffers 992 126))

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 756 bytes --]

Tom Tromey <tom@tromey.com> writes:

> I am running a build of git master from yesterday.
> I tried M-x vc-git-grep.
>
> This successfully runs the grep and puts the output into *grep*.
> However, the highlighting is incorrect and M-x next-error claims there
> are no grep hits.  Using C-c (compile-goto-error) in *grep* also does
> not work.

This is due to [1: 644cdd1aa0].  'git grep' also has a '--null' option,
but the output is a bit different.  Here is a patch to handle it,
although it's not complete: it doesn't take the value of
grep-use-null-filename-separator into account.  Perhaps we should
instead disable this option by default; when I introduced it, I didn't
think about the fact that grep.el is used for not-quite-grep programs as
well.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1907 bytes --]

From 60acf5de6dacb5ea3430a1115ba8c1151a07d526 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Wed, 26 Jul 2017 19:21:55 -0400
Subject: [PATCH v1] Use '--null' for 'git grep' too (Bug#27840)

* lisp/vc/vc-git.el (vc-git-grep): Pass '--null'.
* lisp/progmodes/grep.el (grep--regexp-alist-bin-matcher): Handle the
slightly different output from 'git grep's --null option.
---
 lisp/progmodes/grep.el | 5 +++--
 lisp/vc/vc-git.el      | 4 ++--
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 2ddaf884bc..4723290fbe 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -387,8 +387,9 @@ (defconst grep--regexp-alist-column
 (defconst grep--regexp-alist-bin-matcher
   '("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
 (defconst grep-with-null-regexp-alist
-  `(("^\\([^\0]+\\)\\(\0\\)\\([0-9]+\\):" 1 3 ,grep--regexp-alist-column nil nil
-     (2 '(face unspecified display ":")))
+  `(("^\\([^\0]+\\)\\(\0\\)\\([0-9]+\\)\\([\0:]\\)" 1 3 ,grep--regexp-alist-column nil nil
+     (2 '(face unspecified display ":"))
+     (4 '(face unspecified display ":")))
     ,grep--regexp-alist-bin-matcher)
   "Regexp used to match grep hits.
 See `compilation-error-regexp-alist'.")
diff --git a/lisp/vc/vc-git.el b/lisp/vc/vc-git.el
index cc3e295641..1cdcfb3a5c 100644
--- a/lisp/vc/vc-git.el
+++ b/lisp/vc/vc-git.el
@@ -1433,8 +1433,8 @@ (defun vc-git-grep (regexp &optional files dir)
 	      (setq command nil))
 	(setq dir (file-name-as-directory (expand-file-name dir)))
 	(setq command
-	      (grep-expand-template "git --no-pager grep -n -e <R> -- <F>"
-                                    regexp files))
+	      (grep-expand-template
+               "git --no-pager grep -n --null -e <R> -- <F>" regexp files))
 	(when command
 	  (if (equal current-prefix-arg '(4))
 	      (setq command
-- 
2.11.1


[-- Attachment #3: Type: text/plain, Size: 180 bytes --]



[1: 644cdd1aa0]: 2017-07-19 20:03:00 -0400
  Use grep's --null option (Bug#6843)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=644cdd1aa0a10dbfffa3b9b4c7a97f8cddded0b8

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[Dmitry Gutov]

On 7/27/17 2:32 AM, npostavs@users.sourceforge.net wrote:

> Perhaps we should
> instead disable this option by default; when I introduced it, I didn't
> think about the fact that grep.el is used for not-quite-grep programs as
> well.

Or we can try to think of them as separate. I.e., the patch below also 
works. And if we want to use the --null option for vc-git-grep, we can 
construct a derivative alist in vc-git.el.

Up to you, of course.

diff --git a/lisp/vc/vc-git.el b/lisp/vc/vc-git.el
index cc3e295..f52dc6e 100644
--- a/lisp/vc/vc-git.el
+++ b/lisp/vc/vc-git.el
@@ -1443,7 +1443,8 @@ vc-git-grep
  	    (add-to-history 'grep-history command))))
        (when command
  	(let ((default-directory dir)
-	      (compilation-environment (cons "PAGER=" compilation-environment)))
+	      (compilation-environment (cons "PAGER=" compilation-environment))
+              grep-use-null-filename-separator)
  	  ;; Setting process-setup-function makes exit-message-function work
  	  ;; even when async processes aren't supported.
  	  (compilation-start command 'grep-mode))

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 7/27/17 2:32 AM, npostavs@users.sourceforge.net wrote:
>
>> Perhaps we should
>> instead disable this option by default; when I introduced it, I didn't
>> think about the fact that grep.el is used for not-quite-grep programs as
>> well.
>
> Or we can try to think of them as separate. I.e., the patch below also
> works.

Right, but the question is: are there n+1 other places where we will
need to do this?

> And if we want to use the --null option for vc-git-grep, we can
> construct a derivative alist in vc-git.el.
>
> Up to you, of course.
>
>  	(let ((default-directory dir)
> -	      (compilation-environment (cons "PAGER=" compilation-environment)))
> +	      (compilation-environment (cons "PAGER=" compilation-environment))
> +              grep-use-null-filename-separator)

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[Dmitry Gutov]

On 7/30/17 9:08 PM, npostavs@users.sourceforge.net wrote:

> Right, but the question is: are there n+1 other places where we will
> need to do this?

I don't know.

If we really want the default settings of grep-mode to fit all grep-like 
programs, then the task is suddenly harder, and we do want to default to 
the non-null.

Although, honestly, in that case, I would consider reverting the patch, 
because it increases complexity, yet is unlikely (as non-default) to be 
used by the majority of the users.

Alternatively, one regexp would have to handle both versions, with and 
without null (merging grep-with-null-regexp-alist and 
grep-fallback-regexp-alist together, if it's at all possible).

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Alternatively, one regexp would have to handle both versions, with and
> without null (merging grep-with-null-regexp-alist and
> grep-fallback-regexp-alist together, if it's at all possible).

It is possible, the regexp gets a bit more complicated and the
without-null case can now become confused by files containing NUL bytes,
but that's probably fairly rare and things do get simpler again for
callers.  Seems like the best solution overall.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 6264 bytes --]

From a556574087226edd1f2aa9f3834a5dfb7968080c Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sun, 30 Jul 2017 11:07:01 -0400
Subject: [PATCH v2] Merge null and without-null regexp alists (Bug#27840,
 Bug#27873)

* lisp/progmodes/grep.el (grep--regexp-alist-column)
(grep--regexp-alist-bin-matcher, grep-with-null-regexp-alist)
(grep-fallback-regexp-alist): Remove.
(grep-regexp-alist): Recombine their contents here.
(grep-mode):
* lisp/progmodes/xref.el (xref-collect-matches): Use the variable
`grep-regexp-alist' rather than the function.
---
 lisp/progmodes/grep.el | 86 ++++++++++++++++++++++----------------------------
 lisp/progmodes/xref.el |  2 +-
 2 files changed, 39 insertions(+), 49 deletions(-)

diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 61b82df46a..466b524c79 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -31,7 +31,6 @@
 
 (require 'compile)
 
-
 (defgroup grep nil
   "Run `grep' and display the results."
   :group 'tools
@@ -366,53 +365,44 @@ (defvar grep-last-buffer nil
 Notice that using \\[next-error] or \\[compile-goto-error] modifies
 `compilation-last-buffer' rather than `grep-last-buffer'.")
 
-(defconst grep--regexp-alist-column
-  ;; Calculate column positions (col . end-col) of first grep match on a line
-  (cons
-   (lambda ()
-     (when grep-highlight-matches
-       (let* ((beg (match-end 0))
-              (end (save-excursion (goto-char beg) (line-end-position)))
-              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face)))
-         (when mbeg
-           (- mbeg beg)))))
-   (lambda ()
-     (when grep-highlight-matches
-       (let* ((beg (match-end 0))
-              (end (save-excursion (goto-char beg) (line-end-position)))
-              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face))
-              (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
-         (when mend
-           (- mend beg)))))))
-(defconst grep--regexp-alist-bin-matcher
-  '("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
-(defconst grep-with-null-regexp-alist
-  `(("^\\([^\0]+\\)\\(\0\\)\\([0-9]+\\):" 1 3 ,grep--regexp-alist-column nil nil
-     (2 '(face unspecified display ":")))
-    ,grep--regexp-alist-bin-matcher)
-  "Regexp used to match grep hits.
-See `compilation-error-regexp-alist'.")
-(defconst grep-fallback-regexp-alist
-  `(;; Use a tight regexp to handle weird file names (with colons
-    ;; in them) as well as possible.  E.g., use [1-9][0-9]* rather
-    ;; than [0-9]+ so as to accept ":034:" in file names.
-    ("^\\(.*?[^/\n]\\):[ \t]*\\([1-9][0-9]*\\)[ \t]*:"
-     1 2 ,grep--regexp-alist-column)
-    ,grep--regexp-alist-bin-matcher)
-  "Regexp used to match grep hits when `--null' is not supported.
-See `compilation-error-regexp-alist'.")
-
-(defvaralias 'grep-regex-alist 'grep-with-null-regexp-alist)
-(make-obsolete-variable
- 'grep-regex-alist "Call `grep-regexp-alist' instead." "26.1")
-
 ;;;###autoload
-(defun grep-regexp-alist ()
-  "Return a regexp alist to match grep hits.
-The regexp used depends on `grep-use-null-filename-separator'.
-See `compilation-error-regexp-alist' for format details."
-  (if grep-use-null-filename-separator
-      grep-with-null-regexp-alist grep-fallback-regexp-alist))
+(defconst grep-regexp-alist
+  `((,(concat "^\\(?:"
+              ;; Parse using NUL characters when `--null' is used.
+              ;; Note that we must still assume no newlines in
+              ;; filenames due to "foo: Is a directory." type
+              ;; messages.
+              "\\(?1:[^\0\n]+\\)\\(?3:\0\\)\\(?2:[0-9]+\\):"
+              "\\|"
+              ;; Fallback if `--null' is not used, use a tight regexp
+              ;; to handle weird file names (with colons in them) as
+              ;; well as possible.  E.g., use [1-9][0-9]* rather than
+              ;; [0-9]+ so as to accept ":034:" in file names.
+              "\\(?1:[^\n:]+?[^\n/:]\\):[\t ]*\\(?2:[1-9][0-9]*\\)[\t ]*:"
+              "\\)")
+     1 2
+     ;; Calculate column positions (col . end-col) of first grep match on a line
+     (,(lambda ()
+         (when grep-highlight-matches
+           (let* ((beg (match-end 0))
+                  (end (save-excursion (goto-char beg) (line-end-position)))
+                  (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face)))
+             (when mbeg
+               (- mbeg beg)))))
+      .
+      ,(lambda ()
+         (when grep-highlight-matches
+           (let* ((beg (match-end 0))
+                  (end (save-excursion (goto-char beg) (line-end-position)))
+                  (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face))
+                  (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
+             (when mend
+               (- mend beg))))))
+     nil nil
+     (3 '(face nil display ":")))
+    ("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
+  "Regexp used to match grep hits.
+See `compilation-error-regexp-alist' for format details.")
 
 (defvar grep-first-column 0		; bug#10594
   "Value to use for `compilation-first-column' in grep buffers.")
@@ -783,7 +773,7 @@ (define-compilation-mode grep-mode "Grep"
   (set (make-local-variable 'compilation-error-face)
        grep-hit-face)
   (set (make-local-variable 'compilation-error-regexp-alist)
-       (grep-regexp-alist))
+       grep-regexp-alist)
   ;; compilation-directory-matcher can't be nil, so we set it to a regexp that
   ;; can never match.
   (set (make-local-variable 'compilation-directory-matcher) '("\\`a\\`"))
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index cc9b794c5a..35a5c8862f 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -929,7 +929,7 @@ (defun xref-collect-matches (regexp files dir ignores)
                                      (expand-file-name dir)
                                      ignores))
        (buf (get-buffer-create " *xref-grep*"))
-       (`(,grep-re ,file-group ,line-group . ,_) (car (grep-regexp-alist)))
+       (`(,grep-re ,file-group ,line-group . ,_) (car grep-regexp-alist))
        (status nil)
        (hits nil))
     (with-current-buffer buf
-- 
2.11.1

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[Dmitry Gutov]

On 8/3/17 6:09 AM, npostavs@users.sourceforge.net wrote:
> * lisp/progmodes/grep.el (grep--regexp-alist-column)
> (grep--regexp-alist-bin-matcher, grep-with-null-regexp-alist)
> (grep-fallback-regexp-alist): Remove.
> (grep-regexp-alist): Recombine their contents here.
> (grep-mode):
> * lisp/progmodes/xref.el (xref-collect-matches): Use the variable
> `grep-regexp-alist' rather than the function.

Also semantic/symref/grep.el. I fixed it before in a separate patch, but 
let's keep this update all together.

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Also semantic/symref/grep.el. I fixed it before in a separate patch,
> but let's keep this update all together.

Oops!  Thanks for reminding me.  I also noticed that I had missed
handling 'grep --null --after/before-context=N ...' output correctly.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 7801 bytes --]

From a13241d8c0c126c7690166b51cb507ea11f746d6 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sun, 30 Jul 2017 14:47:05 -0400
Subject: [PATCH v3] Merge null and without-null regexp alists (Bug#27840,
 Bug#27873)

* lisp/progmodes/grep.el (grep-mode-font-lock-keywords): Allow for NUL
characters following filename in grep context lines.
(grep--regexp-alist-column, grep--regexp-alist-bin-matcher)
(grep-with-null-regexp-alist, grep-fallback-regexp-alist): Remove.
(grep-regexp-alist): Recombine their contents here.
(grep-mode):
* lisp/cedet/semantic/symref/grep.el
(semantic-symref-parse-tool-output-one-line):
* lisp/progmodes/xref.el (xref-collect-matches): Use the variable
`grep-regexp-alist' rather than the function.
---
 lisp/cedet/semantic/symref/grep.el |  2 +-
 lisp/progmodes/grep.el             | 90 +++++++++++++++++---------------------
 lisp/progmodes/xref.el             |  2 +-
 3 files changed, 43 insertions(+), 51 deletions(-)

diff --git a/lisp/cedet/semantic/symref/grep.el b/lisp/cedet/semantic/symref/grep.el
index df71508da7..f7c72bfb0b 100644
--- a/lisp/cedet/semantic/symref/grep.el
+++ b/lisp/cedet/semantic/symref/grep.el
@@ -193,7 +193,7 @@ (cl-defmethod semantic-symref-parse-tool-output-one-line ((tool semantic-symref-
   "Parse one line of grep output, and return it as a match list.
 Moves cursor to end of the match."
   (pcase-let
-      ((`(,grep-re ,file-group ,line-group . ,_) (car (grep-regexp-alist))))
+      ((`(,grep-re ,file-group ,line-group . ,_) (car grep-regexp-alist)))
     (cond ((eq (oref tool :resulttype) 'file)
 	   ;; Search for files
 	   (when (re-search-forward "^\\([^\n]+\\)$" nil t)
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 2ddaf884bc..466b524c79 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -31,7 +31,6 @@
 
 (require 'compile)
 
-
 (defgroup grep nil
   "Run `grep' and display the results."
   :group 'tools
@@ -366,53 +365,44 @@ (defvar grep-last-buffer nil
 Notice that using \\[next-error] or \\[compile-goto-error] modifies
 `compilation-last-buffer' rather than `grep-last-buffer'.")
 
-(defconst grep--regexp-alist-column
-  ;; Calculate column positions (col . end-col) of first grep match on a line
-  (cons
-   (lambda ()
-     (when grep-highlight-matches
-       (let* ((beg (match-end 0))
-              (end (save-excursion (goto-char beg) (line-end-position)))
-              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face)))
-         (when mbeg
-           (- mbeg beg)))))
-   (lambda ()
-     (when grep-highlight-matches
-       (let* ((beg (match-end 0))
-              (end (save-excursion (goto-char beg) (line-end-position)))
-              (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face))
-              (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
-         (when mend
-           (- mend beg)))))))
-(defconst grep--regexp-alist-bin-matcher
-  '("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
-(defconst grep-with-null-regexp-alist
-  `(("^\\([^\0]+\\)\\(\0\\)\\([0-9]+\\):" 1 3 ,grep--regexp-alist-column nil nil
-     (2 '(face unspecified display ":")))
-    ,grep--regexp-alist-bin-matcher)
-  "Regexp used to match grep hits.
-See `compilation-error-regexp-alist'.")
-(defconst grep-fallback-regexp-alist
-  `(;; Use a tight regexp to handle weird file names (with colons
-    ;; in them) as well as possible.  E.g., use [1-9][0-9]* rather
-    ;; than [0-9]+ so as to accept ":034:" in file names.
-    ("^\\(.*?[^/\n]\\):[ \t]*\\([1-9][0-9]*\\)[ \t]*:"
-     1 2 ,grep--regexp-alist-column)
-    ,grep--regexp-alist-bin-matcher)
-  "Regexp used to match grep hits when `--null' is not supported.
-See `compilation-error-regexp-alist'.")
-
-(defvaralias 'grep-regex-alist 'grep-with-null-regexp-alist)
-(make-obsolete-variable
- 'grep-regex-alist "Call `grep-regexp-alist' instead." "26.1")
-
 ;;;###autoload
-(defun grep-regexp-alist ()
-  "Return a regexp alist to match grep hits.
-The regexp used depends on `grep-use-null-filename-separator'.
-See `compilation-error-regexp-alist' for format details."
-  (if grep-use-null-filename-separator
-      grep-with-null-regexp-alist grep-fallback-regexp-alist))
+(defconst grep-regexp-alist
+  `((,(concat "^\\(?:"
+              ;; Parse using NUL characters when `--null' is used.
+              ;; Note that we must still assume no newlines in
+              ;; filenames due to "foo: Is a directory." type
+              ;; messages.
+              "\\(?1:[^\0\n]+\\)\\(?3:\0\\)\\(?2:[0-9]+\\):"
+              "\\|"
+              ;; Fallback if `--null' is not used, use a tight regexp
+              ;; to handle weird file names (with colons in them) as
+              ;; well as possible.  E.g., use [1-9][0-9]* rather than
+              ;; [0-9]+ so as to accept ":034:" in file names.
+              "\\(?1:[^\n:]+?[^\n/:]\\):[\t ]*\\(?2:[1-9][0-9]*\\)[\t ]*:"
+              "\\)")
+     1 2
+     ;; Calculate column positions (col . end-col) of first grep match on a line
+     (,(lambda ()
+         (when grep-highlight-matches
+           (let* ((beg (match-end 0))
+                  (end (save-excursion (goto-char beg) (line-end-position)))
+                  (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face)))
+             (when mbeg
+               (- mbeg beg)))))
+      .
+      ,(lambda ()
+         (when grep-highlight-matches
+           (let* ((beg (match-end 0))
+                  (end (save-excursion (goto-char beg) (line-end-position)))
+                  (mbeg (text-property-any beg end 'font-lock-face 'grep-match-face))
+                  (mend (and mbeg (next-single-property-change mbeg 'font-lock-face nil end))))
+             (when mend
+               (- mend beg))))))
+     nil nil
+     (3 '(face nil display ":")))
+    ("^Binary file \\(.+\\) matches$" 1 nil nil 0 1))
+  "Regexp used to match grep hits.
+See `compilation-error-regexp-alist' for format details.")
 
 (defvar grep-first-column 0		; bug#10594
   "Value to use for `compilation-first-column' in grep buffers.")
@@ -451,7 +441,9 @@ (defvar grep-mode-font-lock-keywords
       (2 grep-error-face nil t))
      ;; "filename-linenumber-" format is used for context lines in GNU grep,
      ;; "filename=linenumber=" for lines with function names in "git grep -p".
-     ("^.+?[-=][0-9]+[-=].*\n" (0 grep-context-face)))
+     ("^.+?\\([-=\0]\\)[0-9]+\\([-=]\\).*\n" (0 grep-context-face)
+      (1 (if (eq (char-after (match-beginning 1)) ?\0)
+             `(face nil display ,(match-string 2))))))
    "Additional things to highlight in grep output.
 This gets tacked on the end of the generated expressions.")
 
@@ -781,7 +773,7 @@ (define-compilation-mode grep-mode "Grep"
   (set (make-local-variable 'compilation-error-face)
        grep-hit-face)
   (set (make-local-variable 'compilation-error-regexp-alist)
-       (grep-regexp-alist))
+       grep-regexp-alist)
   ;; compilation-directory-matcher can't be nil, so we set it to a regexp that
   ;; can never match.
   (set (make-local-variable 'compilation-directory-matcher) '("\\`a\\`"))
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index cc9b794c5a..35a5c8862f 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -929,7 +929,7 @@ (defun xref-collect-matches (regexp files dir ignores)
                                      (expand-file-name dir)
                                      ignores))
        (buf (get-buffer-create " *xref-grep*"))
-       (`(,grep-re ,file-group ,line-group . ,_) (car (grep-regexp-alist)))
+       (`(,grep-re ,file-group ,line-group . ,_) (car grep-regexp-alist))
        (status nil)
        (hits nil))
     (with-current-buffer buf
-- 
2.11.1

bug#27840: 26.0.50; vc-git-grep stopped working on git master

[npostavs]

tags 27840 fixed
close 27840 
tags 27873 fixed
close 27873
quit

npostavs@users.sourceforge.net writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> Also semantic/symref/grep.el. I fixed it before in a separate patch,
>> but let's keep this update all together.
>
> Oops!  Thanks for reminding me.  I also noticed that I had missed
> handling 'grep --null --after/before-context=N ...' output correctly.

Pushed to master [1: c2f1830d69].

[1: c2f1830d69]: 2017-08-06 20:25:11 -0400
  Merge null and without-null regexp alists (Bug#27840, Bug#27873)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c2f1830d69f5a5e20dac6fcbf3af4d51afba92bd