Highlighting in grep buffer

Highlighting in grep buffer

[Richard Stallman]

It would be useful to highlight the string in each line that matched
the specified regexp.  It may be nontrivial to parse the command line
that was run, but it would be quite useful, even if it only works
in the most common cases.

RE: Highlighting in grep buffer

[Drew Adams]

FYI - This was part of the Emacs 20 code I sent previously (files
compile-.el and compile+.el).

My code treated the common-case grep patterns using simple regexps (see #3,
below).

My code did *not* try to:

 - treat possible occurrences of grep as part of another command sequence
(e.g. piped)
 - treat grep options: -i for case insensitivity, -E etc. for other grep
flavors etc.
 - treat environment variable substitution

It just grabbed the regexp (quoted or not) from an Emacs M-x grep command,
and used it for highlighting.

Y'all decided that "this is pretty much the same feature that has now been
integrated into Emacs".  I don't have Emacs 21, so I took your word for it;
I know Emacs 21 has lots of great stuff.

I personally find it saves time and eye fatigue to have the search pattern
stand out from the context. The feature is simple, but helpful. If you turn
on Google highlighting, then you will appreciate this same feature for grep.

>From my code:

1. The grep pattern is used here to highlight occurrences in the *grep*
buffer (in function compilation-mode-font-lock-keywords):

   ;; NOTE: No account is taken here of case-insensitivity options to grep
   ;; (e.g. `-i'). This is not generally possible, as different grep's may
use
   ;; different options. Here, only the literal `grep-pattern' string is
   ;; highlighted.
   (and grep-pattern ; Does nothing if this is not a grep command.
        (list
         (list (concat "\\(" (regexp-quote grep-pattern) "\\)")
               1 grep-regexp-face)))

2. The grep pattern is used here to highlight the particulat occurrence in
the line visited by compilation-goto-locus (in function
compilation-goto-locus):

   (set-window-point w (car next-error))
   (set-window-start w (car next-error))
   (highlight-regexp-region (save-excursion (beginning-of-line) (point))
                            (save-excursion (end-of-line) (point))
                            grep-pattern grep-regexp-face)

3. The grep pattern is saved here (in command grep):

(cond (;; Quoted pattern (either "..." or '...')
       (string-match
        (concat
         grep-program
         "[ \t]*\\(-[a-zA-Z]+\\s-+\\)*[ \t]*\\('[^']+'\\|\"[^\"]+\"\\)") ;"
        command-args)
       (setq grep-pattern
             (substring command-args
                        (1+ (match-beginning 2)) (1- (match-end 2)))))
      (;; Unquoted pattern.
       (string-match
        (concat grep-program
                "[ \t]*\\(-[a-zA-Z]+\\s-+\\)*[ \t]*\\([^ \n\t'\"]+\\)") ; "
        command-args)
       (setq grep-pattern
             (substring command-args (match-beginning 2) (match-end 2))))
      (t;; Bad pattern.
       (setq grep-pattern nil))

4. Function compile resets the grep pattern (from last grep) to nil.

HTH,

   Drew

-----Original Message-----
From: emacs-devel-bounces+drew.adams=oracle.com@gnu.org
[mailto:emacs-devel-bounces+drew.adams=oracle.com@gnu.org]On Behalf Of
Richard Stallman
Sent: Monday, April 05, 2004 3:03 PM
To: emacs-devel@gnu.org
Subject: Highlighting in grep buffer


It would be useful to highlight the string in each line that matched
the specified regexp.  It may be nontrivial to parse the command line
that was run, but it would be quite useful, even if it only works
in the most common cases.

Re: Highlighting in grep buffer

[Richard Stallman]

    FYI - This was part of the Emacs 20 code I sent previously (files
    compile-.el and compile+.el).

Could you adapt that specific feature to the new compile.el
and grep.el code?  Then we could install it.

RE: Highlighting in grep buffer

[Drew Adams]

Sorry, I don't have the resources to help on this.

I understand that the Emacs 21 code (compile.el, grep.el) is quite different
from the Emacs 20 code. My changes were quite minor and straightforward (in
the Emacs 20 code), so I think it should be clear how they work. I didn't
try to do anything fancy (e.g. parse grep options) or anything optimal in
terms of performance, so you will likely prefer to rewrite my way of doing
things, in any case.

I offer my code as a suggestion of what might be done, not necessarily a
model of the way to do it.

I'm sorry I can't be more help. If there's any confusion about my code
(unlikely - it's minor-league), I'll be glad to answer questions.

   Drew

-----Original Message-----
From: Richard Stallman [mailto:rms@gnu.org]
Sent: Wednesday, April 07, 2004 10:46 AM
To: Drew Adams
Cc: emacs-devel@gnu.org
Subject: Re: Highlighting in grep buffer


    FYI - This was part of the Emacs 20 code I sent previously (files
    compile-.el and compile+.el).

Could you adapt that specific feature to the new compile.el
and grep.el code?  Then we could install it.

Re: Highlighting in grep buffer

[Juri Linkov]

Isn't something like this a simpler and more reliable solution?

*** emacs/lisp/progmodes/grep.el	11 Mar 2004 22:56:19 -0000	1.10
--- emacs/lisp/progmodes/grep.el	8 Apr 2004 00:37:46 -0000
***************
*** 281,288 ****
--- 281,291 ----
  (defun grep-process-setup ()
    "Setup compilation variables and buffer for `grep'.
  Set up `compilation-exit-message-function' and run `grep-setup-hook'."
+   (setenv "GREP_OPTIONS" "--color=always")
+   (setenv "GREP_COLOR" "01;31")
    (set (make-local-variable 'compilation-exit-message-function)
         (lambda (status code msg)
+          (ansi-color-apply-on-region (point-min) (point-max))
           (if (eq status 'exit)
               (cond ((zerop code)
                      '("finished (matches found)\n" . "matched"))

The matching positions returned by grep could be also used instead of
`highlight-regexp' to highlight substrings in visited source lines.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Highlighting in grep buffer

[Stefan Monnier]

>   (defun grep-process-setup ()
>     "Setup compilation variables and buffer for `grep'.
>   Set up `compilation-exit-message-function' and run `grep-setup-hook'."
> +   (setenv "GREP_OPTIONS" "--color=always")
> +   (setenv "GREP_COLOR" "01;31")
>     (set (make-local-variable 'compilation-exit-message-function)
>          (lambda (status code msg)
> +          (ansi-color-apply-on-region (point-min) (point-max))
>            (if (eq status 'exit)
>                (cond ((zerop code)
>                       '("finished (matches found)\n" . "matched"))

Sounds like a good idea.  The call to ansi-color-apply-on-region should
probably be in the process filter so it's applied on-the-fly rather than at
the end of the command, but that can be changed later on (I think it
requires more changes).

> The matching positions returned by grep could be also used instead of
> `highlight-regexp' to highlight substrings in visited source lines.

How can you get `grep' to return matching positions?


        Stefan

Re: Highlighting in grep buffer

[Richard Stallman]

    Isn't something like this a simpler and more reliable solution?

It is quite simple, but I can't understand it at all.
It needs comments to explain what GREP_COLOR means
and what that value means and why.

It may be less reliable, in that it may not work with non-GNU grep
programs.  Perhaps that is ok; people should install GNU grep.

Re: Highlighting in grep buffer

[Juri Linkov]

Richard Stallman <rms@gnu.org> writes:
> It is quite simple, but I can't understand it at all.
> It needs comments to explain what GREP_COLOR means
> and what that value means and why.

According to grep man page, the environment variable GREP_COLOR
specifies the marker for highlighting, GREP_OPTIONS specifies default
options to be placed in front of any explicit options, and the option
--color surrounds the matching string with the marker found in GREP_COLOR
environment variable.

Actually, markers are ANSI escape sequences like ^[[01;31m and ^[[00m
before and after the matching string.

> It may be less reliable, in that it may not work with non-GNU grep
> programs.  Perhaps that is ok; people should install GNU grep.

Even though this feature will not be available for non-GNU greps,
I can't tell if the GREP_OPTIONS=--colors will cause them to fail.
If so, a new check for the --colors availability could be added to
the grep.el.

But for GNU grep this feature is the most reliable way to find
positions of matching strings.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Highlighting in grep buffer

[Juri Linkov]

>>   (defun grep-process-setup ()
>>     "Setup compilation variables and buffer for `grep'.
>>   Set up `compilation-exit-message-function' and run `grep-setup-hook'."
>> +   (setenv "GREP_OPTIONS" "--color=always")
>> +   (setenv "GREP_COLOR" "01;31")
>>     (set (make-local-variable 'compilation-exit-message-function)
>>          (lambda (status code msg)
>> +          (ansi-color-apply-on-region (point-min) (point-max))
>>            (if (eq status 'exit)
>>                (cond ((zerop code)
>>                       '("finished (matches found)\n" . "matched"))
>
> Sounds like a good idea.  The call to ansi-color-apply-on-region should
> probably be in the process filter so it's applied on-the-fly rather than at
> the end of the command, but that can be changed later on (I think it
> requires more changes).

Yes, calling the ansi-color-apply-on-region would be better from the
process filter, but there is a danger that boundaries of strings that
a process gives to the filter function will be inside an ANSI escape
sequence, which will screw up its fontification.

>> The matching positions returned by grep could be also used instead of
>> `highlight-regexp' to highlight substrings in visited source lines.
>
> How can you get `grep' to return matching positions?

Matching positions are explicitly marked by ANSI escape sequences, so
these positions can be found before translating ANSI escape sequences
into faces.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Highlighting in grep buffer

[Stefan Monnier]

> Yes, calling the ansi-color-apply-on-region would be better from the
> process filter, but there is a danger that boundaries of strings that
> a process gives to the filter function will be inside an ANSI escape
> sequence, which will screw up its fontification.

All process filters have code to deal with this problem.
I.e. it's a non-problem.


        Stefan

Re: Highlighting in grep buffer

[Richard Stallman]

    > It needs comments to explain what GREP_COLOR means
    > and what that value means and why.

    According to grep man page, the environment variable GREP_COLOR
    specifies the marker for highlighting, GREP_OPTIONS specifies default
    options to be placed in front of any explicit options, and the option
    --color surrounds the matching string with the marker found in GREP_COLOR
    environment variable.

Ok, please add comments in the Lisp code with that information.
Don't expect people maintaining Emacs to know such obscure
things about GNU grep!

    > It may be less reliable, in that it may not work with non-GNU grep
    > programs.  Perhaps that is ok; people should install GNU grep.

    Even though this feature will not be available for non-GNU greps,
    I can't tell if the GREP_OPTIONS=--colors will cause them to fail.

It could be they will simply not notice.  Could you arrange to test
this on a few recent modern?

Re: Highlighting in grep buffer

[Miles Bader]

Richard Stallman <rms@gnu.org> writes:
>     > It may be less reliable, in that it may not work with non-GNU grep
>     > programs.  Perhaps that is ok; people should install GNU grep.
> 
>     Even though this feature will not be available for non-GNU greps,
>     I can't tell if the GREP_OPTIONS=--colors will cause them to fail.
> 
> It could be they will simply not notice.  Could you arrange to test
> this on a few recent modern?

It works fine with the native grep on SunOS 4.1.4 and SunOS 5.7 (aka
Solaris 2.7) -- they just ignore it.

-Miles
-- 
Occam's razor split hairs so well, I bought the whole argument!

Re: Highlighting in grep buffer

[Juri Linkov]

"Drew Adams" <drew.adams@oracle.com> writes:
> It's sometimes the case that people use different color schemes
> (light-on-dark vs dark-on-light) in shells (whether inside Emacs or not) and
> in (other) Emacs windows. It's possible, at least.
>
> It would be preferable IMHO for Emacs users to be able to control Emacs
> highlighting 100% within Emacs. They should at least be able to override
> (using, e.g., Emacs user variables) any built-in GNU-grep highlighting
> behavior (e.g. color scheme) that Emacs might take advantage of.
>
> That is, why not have Emacs use its own color variables, which could
> *default* to the grep environment variables - rather than just using the env
> vars directly?

I agree.  A set of ANSI colors is too limited and doesn't match Emacs
colors.  I think we should always set "GREP_COLOR" to the default
value "01;31" and replace the known ANSI sequences containing the
constant default marker in the grep output by a user-defined Emacs face.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Highlighting in grep buffer

[Juri Linkov]

I was thinking about the best solution and it seems that using
font-lock machinery for fontification of grep matches is the simplest
and most consistent with principles which compile.el is based on:
a new rule is added to `grep-mode-font-lock-keywords' which fontifies
the text inside the grep markers and deletes them afterwards.

I also made changes in `compilation-goto-locus' to highlight the
matching string in the source buffer.  The variable `highlight-regexp'
is set to a string found in `compilation-locus' face.  But perhaps
this is not the best way to achieve the goal.  Could someone familiar
with compile.el suggest a better method, for example, something like
setting the `message' property with calculated `col' and `end-col' on
grep markers in `grep-mode-font-lock-keywords' and using them in
`compilation-goto-locus'?

Index: emacs/lisp/progmodes/grep.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/progmodes/grep.el,v
retrieving revision 1.12
diff -u -r1.12 grep.el
--- emacs/lisp/progmodes/grep.el	20 Apr 2004 21:41:50 -0000	1.12
+++ emacs/lisp/progmodes/grep.el	6 May 2004 07:33:14 -0000
@@ -64,6 +64,21 @@
   :version "21.4"
   :group 'grep)
 
+(defcustom grep-highlight-matches t
+  "*Non-nil to use grep markers to highlight matching strings.
+
+Some grep programs are able to surround matching strings with
+special markers in grep output.  Such markers can be used to
+highlight matching strings in grep mode.
+
+This option sets the environment variable GREP_COLOR to specify
+markers for highlighting and GREP_OPTIONS to place the --color
+option in front of any explicit grep options before starting
+the grep."
+  :type 'boolean
+  :version "21.4"
+  :group 'grep)
+
 (defcustom grep-scroll-output nil
   "*Non-nil to scroll the *grep* buffer window as output appears.
 
@@ -244,7 +259,12 @@
      ("^Grep \\(exited abnormally\\) with code \\([0-9]+\\).*"
       (0 '(face nil message nil help-echo nil mouse-face nil) t)
       (1 compilation-warning-face)
-      (2 compilation-line-face)))
+      (2 compilation-line-face))
+     ("\033\\[01;41m\\([^\033\n]*\\)\033\\[00m"
+      (1 'compilation-locus)
+      ((lambda (p))
+       (progn (delete-region (match-end       1) (match-end       0))
+              (delete-region (match-beginning 0) (match-beginning 1))))))
    "Additional things to highlight in grep output.
 This gets tacked on the end of the generated expressions.")
 
@@ -281,6 +301,10 @@
 (defun grep-process-setup ()
   "Setup compilation variables and buffer for `grep'.
 Set up `compilation-exit-message-function' and run `grep-setup-hook'."
+  (when grep-highlight-matches
+    ;; Modify `process-environment' locally bound in `compilation-start'
+    (setenv "GREP_OPTIONS" (concat (getenv "GREP_OPTIONS") " --color=always"))
+    (setenv "GREP_COLOR" "01;41"))
   (set (make-local-variable 'compilation-exit-message-function)
        (lambda (status code msg)
 	 (if (eq status 'exit)

Index: emacs/lisp/progmodes/compile.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/progmodes/compile.el,v
retrieving revision 1.316
diff -u -r1.316 compile.el
--- emacs/lisp/progmodes/compile.el	2 May 2004 20:45:36 -0000	1.316
+++ emacs/lisp/progmodes/compile.el	6 May 2004 07:33:16 -0000
@@ -1541,10 +1541,17 @@
 	 (w (or (get-buffer-window (marker-buffer msg) 'visible)
 		;; Pop up a window.
 		(display-buffer (marker-buffer msg))))
+         p
 	 (highlight-regexp (with-current-buffer (marker-buffer msg)
 			     ;; also do this while we change buffer
 			     (compilation-set-window w msg)
-			     compilation-highlight-regexp)))
+                             (if (setq p (text-property-any
+                                          (point) (line-end-position)
+                                          'face 'compilation-locus))
+                                 (buffer-substring-no-properties
+                                  p (next-single-char-property-change
+                                     p 'face))
+                               compilation-highlight-regexp))))
     (compilation-set-window-height w)
 
     (when (and highlight-regexp
@@ -1552,8 +1559,9 @@
       (unless compilation-highlight-overlay
 	(setq compilation-highlight-overlay
 	      (make-overlay (point-min) (point-min)))
-	(overlay-put compilation-highlight-overlay 'face 'region))
+	(overlay-put compilation-highlight-overlay 'face 'compilation-locus))
       (with-current-buffer (marker-buffer mk)
+        (setq p nil)
 	(save-excursion
 	  (end-of-line)
 	  (let ((end (point)))
@@ -1561,11 +1569,42 @@
 	    (if (and (stringp highlight-regexp)
 		     (re-search-forward highlight-regexp end t))
 		(progn
-		  (goto-char (match-beginning 0))
-		  (move-overlay compilation-highlight-overlay (match-beginning 0) (match-end 0)))
-	      (move-overlay compilation-highlight-overlay (point) end))
-	    (sit-for 0.5)
-	    (delete-overlay compilation-highlight-overlay)))))))
+		  (goto-char (setq p (match-beginning 0)))
+		  (move-overlay compilation-highlight-overlay (match-beginning 0) (match-end 0) (current-buffer)))
+	      (move-overlay compilation-highlight-overlay (point) end (current-buffer)))
+	    (sit-for 0.5)
+	    (delete-overlay compilation-highlight-overlay)))
+        (if p (goto-char p))))))
+
+(defface compilation-locus
+  '((t (:inherit region)))
+  "Face used to highlight compilation locus."
+  :group 'font-lock-highlighting-faces
+  :version "21.4")

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Highlighting in grep buffer

[Stefan Monnier]

> I was thinking about the best solution and it seems that using
> font-lock machinery for fontification of grep matches is the simplest
> and most consistent with principles which compile.el is based on:
> a new rule is added to `grep-mode-font-lock-keywords' which fontifies
> the text inside the grep markers and deletes them afterwards.

Agreed, but your current code will fail to re-highlight the matches if the
buffer is refontified.
We should either keep the markers (and make them invisible) or at least
place the face on the `font-lock-face' property rather than on the
`face' property (I'm not sure if this second method works "by design"
or "by accident", tho).

> I also made changes in `compilation-goto-locus' to highlight the
> matching string in the source buffer.  The variable `highlight-regexp'
> is set to a string found in `compilation-locus' face.  But perhaps
> this is not the best way to achieve the goal.

Seems pretty ugly indeed.

> Could someone familiar with compile.el suggest a better method, for
> example, something like setting the `message' property with calculated
> `col' and `end-col' on grep markers in `grep-mode-font-lock-keywords' and
> using them in `compilation-goto-locus'?

I'd rather keep this for a future version of Emacs.


        Stefan

Re: Highlighting in grep buffer

[Miles Bader]

On Thu, May 06, 2004 at 10:39:02AM -0400, Stefan Monnier wrote:
> We should either keep the markers (and make them invisible) or at least
> place the face on the `font-lock-face' property rather than on the
> `face' property (I'm not sure if this second method works "by design"
> or "by accident", tho).

The font-lock-face stuff is an intentional feature designed for modes that
want to support `M-x font-lock', but don't want to bother with the rest of
the font-lock mechanism; it should work in conjunction with traditional
font-lock use too.

-Miles
-- 
"1971 pickup truck; will trade for guns"

Re: Highlighting in grep buffer

[Juri Linkov]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> I was thinking about the best solution and it seems that using
>> font-lock machinery for fontification of grep matches is the simplest
>> and most consistent with principles which compile.el is based on:
>> a new rule is added to `grep-mode-font-lock-keywords' which fontifies
>> the text inside the grep markers and deletes them afterwards.
>
> Agreed, but your current code will fail to re-highlight the matches if the
> buffer is refontified.
> We should either keep the markers (and make them invisible) or at least
> place the face on the `font-lock-face' property rather than on the
> `face' property (I'm not sure if this second method works "by design"
> or "by accident", tho).

>From these two variants using `font-lock-face' seems better than
making markers invisible, because often hidden text is too confusing
for users.

>> Could someone familiar with compile.el suggest a better method, for
>> example, something like setting the `message' property with calculated
>> `col' and `end-col' on grep markers in `grep-mode-font-lock-keywords' and
>> using them in `compilation-goto-locus'?
>
> I'd rather keep this for a future version of Emacs.

Since the new compile.el is a complete rewrite, would it be better to
consolidate all major changes in one release?

-- 
Juri Linkov
http://www.jurta.org/emacs/