bug#59888: [PATCH] Add 'grep-heading-mode'

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

Tags: patch

This simulates the --heading option of certain grep-like tools.  The
output is like this:

--8<---------------cut here---------------start------------->8---
-*- mode: grep; default-directory: "~/Projects/emacs/" -*-
Grep started at Wed Dec  7 18:43:40

find [...] -exec grep --color=auto -i -nH --null -e test \{\} +

./nextstep/Makefile.in
104:	find ${ns_appdir} -exec test \! -e {} \; -ls

./nextstep/INSTALL
12:Tested on GNU/Linux, may work on other systems.

./nextstep/Makefile
104:	find ${ns_appdir} -exec test \! -e {} \; -ls

./nextstep/README
62:  10.6.8 (Snow Leopard) to the latest official release.
86:* This allows other Emacs developers to test their changes on the NS
--8<---------------cut here---------------end--------------->8---


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-grep-heading-mode.patch --]
[-- Type: text/patch, Size: 4316 bytes --]

From 2247f006845000032fedc3dda9a073b14043a270 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Wed, 7 Dec 2022 18:44:07 +0100
Subject: [PATCH] Add 'grep-heading-mode'

New minor mode to subdivide grep output into sections, as in the
'--heading' option of certain grep-like programs.

* lisp/progmodes/grep.el (grep-heading-regexp): New user option.
(grep-heading): New face.
(grep--heading-format, grep--current-heading, grep--heading-filter):
Filter function for grep processes and supporting variables.
(grep-heading-mode): New minor mode.
---
 etc/NEWS               |  8 +++++
 lisp/progmodes/grep.el | 71 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 3eeef0ab4c..54c3b2447a 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -41,6 +41,14 @@ connection.
 \f
 * Changes in Specialized Modes and Packages in Emacs 30.1
 
+** Compile
+
+*** New minor mode 'grep-heading-mode'.
+In this mode, the grep output is split into sections, one for each
+file, instead of having file names prefixed to each line.  It is
+equivalent to the --heading option of some tools such as 'git grep'
+and 'rg.
+
 ** VC
 
 ---
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 2446e86abb..132f905734 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -30,6 +30,7 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib))
+(eval-when-compile (require 'rx))
 (require 'compile)
 
 (defgroup grep nil
@@ -1407,6 +1408,76 @@ grep-file-at-point
 ;;;###autoload
 (defalias 'rzgrep #'zrgrep)
 
+;;; Headings mode
+(defcustom grep-heading-regexp
+  (rx bol
+      (or
+       (seq "Grep" (* (any "/a-zA-Z")) " "
+            (or "started" "finished" "exited" "interrupt" "killed" "terminated")
+            (* (not (any "\0\n"))))
+       (seq
+        (group-n 2
+          (group-n 1 (+? any))
+          (any "\0-:="))
+        (+ digit)
+        (any "-:="))))
+  "Regexp used to create headings from grep output lines.
+It should be anchored at beginning of line.  The first capture
+group, if present, should match the heading associated to the
+line.  The buffer range of the second capture, if present, is
+made invisible (presumably because displaying it would be
+redundant)."
+  :type 'regexp
+  :version "30.1")
+
+(defface grep-heading '((t :inherit font-lock-function-name-face))
+  "Face of headings when using `grep-heading-mode'.")
+
+(defvar grep--heading-format
+  #("\n%s\n" 1 3 (font-lock-face grep-heading outline-level 1))
+  "Format string of grep headings.
+This is passed to `format' with one argument, the text of the
+first capture group of `grep-heading-regexp'.")
+
+(defvar-local grep--current-heading nil
+  "Used by `grep--heading-filter' to keep track of the current heading.")
+
+(defun grep--heading-filter ()
+  "Filter function to add headings to output of a grep process."
+  (save-excursion
+    (let ((bound (copy-marker (pos-bol))))
+      (goto-char compilation-filter-start)
+      (forward-line 0)
+      (while (re-search-forward grep-heading-regexp bound t)
+        (let ((heading (match-string-no-properties 1))
+              (start (match-beginning 2))
+              (end (match-end 2)))
+          (when start
+            (put-text-property start end 'invisible t))
+          (when (and heading (not (equal heading grep--current-heading)))
+            (save-excursion
+              (forward-line 0)
+              (insert-before-markers (format grep--heading-format heading)))
+            (setq grep--current-heading heading)))))))
+
+;;;###autoload
+(define-minor-mode grep-heading-mode
+  "Subdivide grep output into sections, one per file."
+  :interactive 'grep-mode
+  (if (not grep-heading-mode)
+      (recompile)
+    (save-excursion
+      (save-restriction
+        (widen)
+        (let ((inhibit-read-only t)
+              (compilation-filter-start (point-min)))
+          (goto-char (point-max))
+          (grep--heading-filter))))
+    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
+    (setq-local outline-search-function #'outline-search-level
+                outline-level (lambda () (get-text-property
+                                          (point) 'outline-level)))))
+
 (provide 'grep)
 
 ;;; grep.el ends here
-- 
2.38.1


[-- Attachment #3: Type: text/plain, Size: 539 bytes --]


Some comments:

1. It's necessary to insert text into the grep buffer, which is a bit
   concerning because 'compilation-filter-start' is not a marker, just a
   number.  This could cause other filter functions to get confused.
   This is why I'm adding to 'compilation-filter-hook at a higher depth.
   Perhaps there are better approaches here.

2. One could get rid of the awkward first alternative of
   `grep-heading-regexp' if there was a way to distringuish the grep
   process output from the header and footer added by grep-mode.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Wed, 07 Dec 2022 18:57:37 +0100
> 
> +(defcustom grep-heading-regexp
> +  (rx bol
> +      (or
> +       (seq "Grep" (* (any "/a-zA-Z")) " "
> +            (or "started" "finished" "exited" "interrupt" "killed" "terminated")

Cannot these be translated to languages other than English in
localized versions of Grep?

> 2. One could get rid of the awkward first alternative of
>    `grep-heading-regexp' if there was a way to distringuish the grep
>    process output from the header and footer added by grep-mode.

You could put a special text property on the latter, no?

bug#59888: [PATCH] Add 'grep-heading-mode'

[Stefan Kangas]

Augusto Stoffel <arstoffel@gmail.com> writes:

> From 2247f006845000032fedc3dda9a073b14043a270 Mon Sep 17 00:00:00 2001
> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Wed, 7 Dec 2022 18:44:07 +0100
> Subject: [PATCH] Add 'grep-heading-mode'
>
> New minor mode to subdivide grep output into sections, as in the
> '--heading' option of certain grep-like programs.
>
> * lisp/progmodes/grep.el (grep-heading-regexp): New user option.
> (grep-heading): New face.
> (grep--heading-format, grep--current-heading, grep--heading-filter):
> Filter function for grep processes and supporting variables.
> (grep-heading-mode): New minor mode.

How about making this into a defcustom instead of a minor mode?

I'm thinking that something like

    (setopt grep-use-headings t)

is a slightly simpler interface to use than

    (add-hook 'grep-mode-hook 'grep-heading-mode)

I also think the former plays a bit better with customize.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

On Wed,  7 Dec 2022 at 20:14, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Date: Wed, 07 Dec 2022 18:57:37 +0100
>> 
>> +(defcustom grep-heading-regexp
>> +  (rx bol
>> +      (or
>> +       (seq "Grep" (* (any "/a-zA-Z")) " "
>> +            (or "started" "finished" "exited" "interrupt" "killed" "terminated")
>
> Cannot these be translated to languages other than English in
> localized versions of Grep?

AFAICT these messages are generated by Emacs itself.

We could also use "^Grep[/a-zA-Z]* [^\0t\]*$" here, which has a slightly
higher change of being confused with actual grep output, but only when
not using grep --null.

>> 2. One could get rid of the awkward first alternative of
>>    `grep-heading-regexp' if there was a way to distringuish the grep
>>    process output from the header and footer added by grep-mode.
>
> You could put a special text property on the latter, no?

That's an option and it would be useful for other extensions as well.
So we would define something like

--8<---------------cut here---------------start------------->8---
(defun compilation-insert-note (&rest args)
  (let ((start (point)))
    (apply #'insert args)
    (put-text-property start (point) 'compilation-note t)))
--8<---------------cut here---------------end--------------->8---

and use throughout compile.el, right?

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

On Wed,  7 Dec 2022 at 16:19, Stefan Kangas wrote:

> Augusto Stoffel <arstoffel@gmail.com> writes:
>
>> From 2247f006845000032fedc3dda9a073b14043a270 Mon Sep 17 00:00:00 2001
>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Date: Wed, 7 Dec 2022 18:44:07 +0100
>> Subject: [PATCH] Add 'grep-heading-mode'
>>
>> New minor mode to subdivide grep output into sections, as in the
>> '--heading' option of certain grep-like programs.
>>
>> * lisp/progmodes/grep.el (grep-heading-regexp): New user option.
>> (grep-heading): New face.
>> (grep--heading-format, grep--current-heading, grep--heading-filter):
>> Filter function for grep processes and supporting variables.
>> (grep-heading-mode): New minor mode.
>
> How about making this into a defcustom instead of a minor mode?
>
> I'm thinking that something like
>
>     (setopt grep-use-headings t)
>
> is a slightly simpler interface to use than
>
>     (add-hook 'grep-mode-hook 'grep-heading-mode)
>
> I also think the former plays a bit better with customize.

This is at the same time less customizable in the sense that something
like

  (add-hook 'grep-mode-hook
    (lambda () (when condition) (setq grep-use-headings t)))

will not work (I think), while conditionally activating a minor mode
certainly works.  But I'm open to both approaches.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Mattias Engdegård]

No argument about the idea (good to me) or implementation but minor points regarding the regexp:

> +          (group-n 1 (+? any))

Very much prefer `nonl` or `not-newline` to `any`, which is only there for compatibility (shouldn't have been included in the first place).

> +          (any "\0-:="))

Is the range (ASCII 00..1A) intended here, or should that be a literal hyphen?
If the former, it should probably be made more explicit since that range includes all sorts of numbers, symbols and control chars and it kind of looks like it may be a mistake.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

On Thu,  8 Dec 2022 at 10:57, Mattias Engdegård wrote:

> No argument about the idea (good to me) or implementation but minor points regarding the regexp:
>
>> +          (group-n 1 (+? any))
>
> Very much prefer `nonl` or `not-newline` to `any`, which is only there for compatibility (shouldn't have been included in the first place).

Hum, not-newline is too long, so this leaves an option between the
slightly deceiving and the quite obscure.

Okay, now I see `any' is not even documented.

>> +          (any "\0-:="))
>
> Is the range (ASCII 00..1A) intended here, or should that be a literal hyphen?
> If the former, it should probably be made more explicit since that
> range includes all sorts of numbers, symbols and control chars and it
> kind of looks like it may be a mistake.

The literal hyphen, which prefixes context lines in `grep -C <n>'.
Thanks for the heads up.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Mattias Engdegård]

8 dec. 2022 kl. 11.28 skrev Augusto Stoffel <arstoffel@gmail.com>:

>> Very much prefer `nonl` or `not-newline` to `any`, which is only there for compatibility (shouldn't have been included in the first place).
> 
> Hum, not-newline is too long, so this leaves an option between the
> slightly deceiving and the quite obscure.

Actually `nonl` isn't too bad once you get used to it. Pronounce it 'nonnel', stress on the first syllable.

> Okay, now I see `any' is not even documented.

Yes, it was inherited from the defunct `sregex` package, in hindsight a clear mistake.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 59888@debbugs.gnu.org
> Date: Thu, 08 Dec 2022 09:59:46 +0100
> 
> On Wed,  7 Dec 2022 at 20:14, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel@gmail.com>
> >> Date: Wed, 07 Dec 2022 18:57:37 +0100
> >> 
> >> +(defcustom grep-heading-regexp
> >> +  (rx bol
> >> +      (or
> >> +       (seq "Grep" (* (any "/a-zA-Z")) " "
> >> +            (or "started" "finished" "exited" "interrupt" "killed" "terminated")
> >
> > Cannot these be translated to languages other than English in
> > localized versions of Grep?
> 
> AFAICT these messages are generated by Emacs itself.

All of them?

> We could also use "^Grep[/a-zA-Z]* [^\0t\]*$" here

The [/a-zA-Z] part is still ASCII-only, and I believe the "/" part
means you want to support file names? then for Windows we also need
the colon ':'.

> >> 2. One could get rid of the awkward first alternative of
> >>    `grep-heading-regexp' if there was a way to distringuish the grep
> >>    process output from the header and footer added by grep-mode.
> >
> > You could put a special text property on the latter, no?
> 
> That's an option and it would be useful for other extensions as well.
> So we would define something like
> 
> --8<---------------cut here---------------start------------->8---
> (defun compilation-insert-note (&rest args)
>   (let ((start (point)))
>     (apply #'insert args)
>     (put-text-property start (point) 'compilation-note t)))
> --8<---------------cut here---------------end--------------->8---
> 
> and use throughout compile.el, right?

Yes, something like this.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Juri Linkov]

>> How about making this into a defcustom instead of a minor mode?
>>
>> I'm thinking that something like
>>
>>     (setopt grep-use-headings t)
>>
>> is a slightly simpler interface to use than
>>
>>     (add-hook 'grep-mode-hook 'grep-heading-mode)
>>
>> I also think the former plays a bit better with customize.
>
> This is at the same time less customizable in the sense that something
> like
>
>   (add-hook 'grep-mode-hook
>     (lambda () (when condition) (setq grep-use-headings t)))
>
> will not work (I think), while conditionally activating a minor mode
> certainly works.  But I'm open to both approaches.

Why not?  (setq-local grep-use-headings t) definitely should do
the right thing.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Juri Linkov]

> Subject: [PATCH] Add 'grep-heading-mode'
>
> New minor mode to subdivide grep output into sections, as in the
> '--heading' option of certain grep-like programs.
>
> * lisp/progmodes/grep.el (grep-heading-regexp): New user option.
> (grep-heading): New face.
> (grep--heading-format, grep--current-heading, grep--heading-filter):
> Filter function for grep processes and supporting variables.
> (grep-heading-mode): New minor mode.

Thanks, this would be a great addition.

> +(defface grep-heading '((t :inherit font-lock-function-name-face))
> +  "Face of headings when using `grep-heading-mode'.")

I suggest to keep the existing color scheme, thus inheriting from
compilation-info.  This is the same color used by both grep.el and
xref.el by default.

> +(defvar grep--heading-format
> +  #("\n%s\n" 1 3 (font-lock-face grep-heading outline-level 1))

xref.el doesn't insert extra newlines between file sections.
It would be nice to keep the output closer to the existing
output in xref buffers.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

On Fri,  9 Dec 2022 at 09:23, Juri Linkov wrote:

>>> How about making this into a defcustom instead of a minor mode?
>>>
>>> I'm thinking that something like
>>>
>>>     (setopt grep-use-headings t)
>>>
>>> is a slightly simpler interface to use than
>>>
>>>     (add-hook 'grep-mode-hook 'grep-heading-mode)
>>>
>>> I also think the former plays a bit better with customize.
>>
>> This is at the same time less customizable in the sense that something
>> like
>>
>>   (add-hook 'grep-mode-hook
>>     (lambda () (when condition) (setq grep-use-headings t)))
>>
>> will not work (I think), while conditionally activating a minor mode
>> certainly works.  But I'm open to both approaches.
>
> Why not?  (setq-local grep-use-headings t) definitely should do
> the right thing.

The variable has to be set at the moment `grep-mode' runs, while a minor
mode could be set at a later point.  But I agree a variable is
sufficient here, so I will make this change.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Augusto Stoffel]

On Fri,  9 Dec 2022 at 09:28, Juri Linkov wrote:

>> +(defface grep-heading '((t :inherit font-lock-function-name-face))
>> +  "Face of headings when using `grep-heading-mode'.")
>
> I suggest to keep the existing color scheme, thus inheriting from
> compilation-info.  This is the same color used by both grep.el and
> xref.el by default.

Yes, I came to the same conclusion in the meanwhile.

>> +(defvar grep--heading-format
>> +  #("\n%s\n" 1 3 (font-lock-face grep-heading outline-level 1))
>
> xref.el doesn't insert extra newlines between file sections.
> It would be nice to keep the output closer to the existing
> output in xref buffers.

Okay, I agree with your reasoning and will make the change.  However, I
must say find the result less appealing.  Note that, because of the lack
of newlines, xref-file-header was naturally chosen as a heavier face.

But it's all customizable anyway.

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 659 bytes --]

Here is an updated patch for the “grep headings” feature.

As discussed before, I introduced a text property so that one can tell
without guessing which parts of the compilation buffer are not coming
from the external process.  This seems to supersede the
'compilation-header-end' property introduced by Lars in commit
07f748da43, so I replaced its uses by the new 'compilation-aside'
property.  I could easily revert that, but it seemed reasonable to
uniformize things in this case.

I've also incorporated all other suggestions from other messages.  (And
Juri, nevermind what I said about some faces, it only applies to the
Modus theme.)


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Introduce-compilation-aside-text-property.patch --]
[-- Type: text/x-patch, Size: 3896 bytes --]

From c593fc94f2289d4bdcb61835eaf11b0fe393a0f5 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Thu, 8 Dec 2022 21:05:10 +0100
Subject: [PATCH 1/2] Introduce 'compilation-aside' text property

It is meant to mark parts of compilation buffers which do not
correspond to process output.

* lisp/progmodes/compile.el (compilation-insert-aside):  New function.
(compilation-start, compilation-handle-exit): Use it.
(compilation--ensure-parse) Rely on 'compilation-aside' property
instead of 'compilation-header-end'
---
 lisp/progmodes/compile.el | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index e8ada9388e..2d22501017 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1675,7 +1675,7 @@ compilation--ensure-parse
             (set-marker (make-marker)
                         (save-excursion
                           (goto-char (point-min))
-                          (text-property-search-forward 'compilation-header-end)
+                          (text-property-search-forward 'compilation-aside)
                           ;; If we have no end marker, this will be
                           ;; `point-min' still.
                           (point)))))
@@ -1823,6 +1823,14 @@ compilation--update-in-progress-mode-line
   ;; buffers when it changes from nil to non-nil or vice-versa.
   (unless compilation-in-progress (force-mode-line-update t)))
 
+(defun compilation-insert-aside (&rest args)
+  "Insert ARGS at point, adding the `compilation-aside' text property.
+This property is used to distinguish output of the compilation
+process from additional information inserted by Emacs."
+  (let ((start (point)))
+    (apply #'insert args)
+    (put-text-property start (point) 'compilation-aside t)))
+
 ;;;###autoload
 (defun compilation-start (command &optional mode name-function highlight-regexp
                                   continue)
@@ -1944,17 +1952,16 @@ compilation-start
             (setq-local compilation-auto-jump-to-next t))
         (when (zerop (buffer-size))
 	  ;; Output a mode setter, for saving and later reloading this buffer.
-	  (insert "-*- mode: " name-of-mode
-		  "; default-directory: "
-                  (prin1-to-string (abbreviate-file-name default-directory))
-		  " -*-\n"))
-        (insert (format "%s started at %s\n\n"
-			mode-name
-			(substring (current-time-string) 0 19))
-		command "\n")
-        ;; Mark the end of the header so that we don't interpret
-        ;; anything in it as an error.
-        (put-text-property (1- (point)) (point) 'compilation-header-end t)
+	  (compilation-insert-aside
+           "-*- mode: " name-of-mode
+           "; default-directory: "
+           (prin1-to-string (abbreviate-file-name default-directory))
+	   " -*-\n"))
+        (compilation-insert-aside
+         (format "%s started at %s\n\n"
+                 mode-name
+		 (substring (current-time-string) 0 19))
+	 command "\n")
 	(setq thisdir default-directory))
       (set-buffer-modified-p nil))
     ;; Pop up the compilation buffer.
@@ -2436,13 +2443,13 @@ compilation-handle-exit
 	(cur-buffer (current-buffer)))
     ;; Record where we put the message, so we can ignore it later on.
     (goto-char omax)
-    (insert ?\n mode-name " " (car status))
+    (compilation-insert-aside ?\n mode-name " " (car status))
     (if (and (numberp compilation-window-height)
 	     (zerop compilation-window-height))
 	(message "%s" (cdr status)))
     (if (bolp)
 	(forward-char -1))
-    (insert " at " (substring (current-time-string) 0 19))
+    (compilation-insert-aside " at " (substring (current-time-string) 0 19))
     (goto-char (point-max))
     ;; Prevent that message from being recognized as a compilation error.
     (add-text-properties omax (point)
-- 
2.38.1


[-- Attachment #3: 0002-New-user-option-grep-use-headings.patch --]
[-- Type: text/x-patch, Size: 6324 bytes --]

From 549187135df21702210050e873210bd200612f96 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Wed, 7 Dec 2022 18:44:07 +0100
Subject: [PATCH 2/2] New user option 'grep-use-headings'

* lisp/progmodes/grep.el (grep-heading-regexp): New user option.
(grep-heading): New face.
(grep--heading-format, grep--heading-state, grep--heading-filter):
Filter function for grep processes and supporting variables.
(grep-use-headings): New user option.
(grep-mode): Use the above, if applicable.
---
 etc/NEWS                          |  8 ++++
 lisp/progmodes/grep.el            | 69 +++++++++++++++++++++++++++++++
 test/lisp/progmodes/grep-tests.el | 14 +++++++
 3 files changed, 91 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 3eeef0ab4c..4ad6e76687 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -41,6 +41,14 @@ connection.
 \f
 * Changes in Specialized Modes and Packages in Emacs 30.1
 
+** Compile
+
+*** New user option 'grep-use-headings'.
+When non-nil, the grep output is split into sections, one for each
+file, instead of having file names prefixed to each line.  It is
+equivalent to the --heading option of some tools such as 'git grep'
+and 'rg.
+
 ** VC
 
 ---
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 2446e86abb..d6981c5951 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -30,6 +30,7 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib))
+(eval-when-compile (require 'rx))
 (require 'compile)
 
 (defgroup grep nil
@@ -457,6 +458,35 @@ grep-search-path
   :type '(repeat (choice (const :tag "Default" nil)
 			 (string :tag "Directory"))))
 
+(defcustom grep-use-headings nil
+  "If non-nil, subdivide grep output into sections, one per file."
+  :type 'boolean
+  :version "30.1")
+
+(defcustom grep-heading-regexp
+  (rx bol
+      (or
+       (group-n 2
+         (group-n 1 (+ (not (any 0 ?\n))))
+         0)
+       (group-n 2
+        (group-n 1 (+? nonl))
+        (any ?: ?- ?=)))
+      (+ digit)
+      (any ?: ?- ?=))
+  "Regexp used to create headings from grep output lines.
+It should be anchored at beginning of line.  The first capture
+group, if present, should match the heading associated to the
+line.  The buffer range of the second capture, if present, is
+made invisible (presumably because displaying it would be
+redundant)."
+  :type 'regexp
+  :version "30.1")
+
+(defface grep-heading `((t :inherit ,grep-hit-face))
+  "Face of headings when using `grep-heading-mode'."
+  :version "30.1")
+
 (defvar grep-find-abbreviate-properties
   (let ((ellipsis (if (char-displayable-p ?…) "[…]" "[...]"))
         (map (make-sparse-keymap)))
@@ -612,6 +642,40 @@ grep-filter
         (while (re-search-forward "\033\\[[0-9;]*[mK]" end 1)
           (replace-match "" t t))))))
 
+(defvar grep--heading-format
+  (eval-when-compile
+    (let ((title (propertize "%s"
+                             'font-lock-face 'grep-heading
+                             'outline-level 1)))
+      (propertize (concat title "\n") 'compilation-aside t)))
+  "Format string of grep headings.
+This is passed to `format' with one argument, the text of the
+first capture group of `grep-heading-regexp'.")
+
+(defvar-local grep--heading-state nil
+  "Variable to keep track of the `grep--heading-filter' state.")
+
+(defun grep--heading-filter ()
+  "Filter function to add headings to output of a grep process."
+  (unless grep--heading-state
+    (setq grep--heading-state (cons (point-min-marker) nil)))
+  (save-excursion
+    (let ((limit (car grep--heading-state)))
+      ;; Move point to the old limit and update limit marker.
+      (move-marker limit (prog1 (pos-bol) (goto-char limit)))
+      (while (re-search-forward grep-heading-regexp limit t)
+        (unless (get-text-property (point) 'compilation-aside)
+          (let ((heading (match-string-no-properties 1))
+                (start (match-beginning 2))
+                (end (match-end 2)))
+            (when start
+              (put-text-property start end 'invisible t))
+            (when (and heading (not (equal heading (cdr grep--heading-state))))
+              (save-excursion
+                (forward-line 0)
+                (insert-before-markers (format grep--heading-format heading)))
+              (setf (cdr grep--heading-state) heading))))))))
+
 (defun grep-probe (command args &optional func result)
   (let (process-file-side-effects)
     (equal (condition-case nil
@@ -906,6 +970,11 @@ grep-mode
   (add-function :filter-return (local 'kill-transform-function)
                 (lambda (string)
                   (string-replace "\0" ":" string)))
+  (when grep-use-headings
+    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
+    (setq-local outline-search-function #'outline-search-level
+                outline-level (lambda () (get-text-property
+                                          (point) 'outline-level))))
   (add-hook 'compilation-filter-hook #'grep-filter nil t))
 
 (defun grep--save-buffers ()
diff --git a/test/lisp/progmodes/grep-tests.el b/test/lisp/progmodes/grep-tests.el
index 101052c5ad..51f4606639 100644
--- a/test/lisp/progmodes/grep-tests.el
+++ b/test/lisp/progmodes/grep-tests.el
@@ -66,4 +66,18 @@ grep-tests--rgrep-abbreviate-properties-windows-nt-sh-semantics
     (cl-letf (((symbol-function 'w32-shell-dos-semantics) #'ignore))
       (grep-tests--check-rgrep-abbreviation))))
 
+(ert-deftest grep-tests--grep-heading-regexp-without-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "filename%c123%ctext" sep sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "filename"))
+      (should (equal (match-string 2 string) (format "filename%c" sep))))))
+
+(ert-deftest grep-tests--grep-heading-regexp-with-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "funny:0:filename%c123%ctext" 0 sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "funny:0:filename"))
+      (should (equal (match-string 2 string) "funny:0:filename\0")))))
+
 ;;; grep-tests.el ends here
-- 
2.38.1

bug#59888: [PATCH] Add 'grep-use-headings'

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  59888@debbugs.gnu.org, Eli
>  Zaretskii <eliz@gnu.org>, Lars Ingebrigtsen <larsi@gnus.org>
> Date: Fri, 09 Dec 2022 13:18:02 +0100
> 
> As discussed before, I introduced a text property so that one can tell
> without guessing which parts of the compilation buffer are not coming
> from the external process.  This seems to supersede the
> 'compilation-header-end' property introduced by Lars in commit
> 07f748da43, so I replaced its uses by the new 'compilation-aside'
> property.  I could easily revert that, but it seemed reasonable to
> uniformize things in this case.

Thanks, but please find a better name for this property.  Something
like compilation-meta-data, perhaps?

> --- a/etc/NEWS
> +++ b/etc/NEWS
> @@ -41,6 +41,14 @@ connection.
>  \f
>  * Changes in Specialized Modes and Packages in Emacs 30.1
>  
> +** Compile
> +
> +*** New user option 'grep-use-headings'.
> +When non-nil, the grep output is split into sections, one for each
                     ^^^^
"Grep", capitalized.  Or maybe even "the output of Grep".

> +file, instead of having file names prefixed to each line.  It is
> +equivalent to the --heading option of some tools such as 'git grep'
> +and 'rg.
       ^^^
'rg'

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

On Fri,  9 Dec 2022 at 21:36, Eli Zaretskii wrote:

>> As discussed before, I introduced a text property so that one can tell
>> without guessing which parts of the compilation buffer are not coming
>> from the external process.  This seems to supersede the
>> 'compilation-header-end' property introduced by Lars in commit
>> 07f748da43, so I replaced its uses by the new 'compilation-aside'
>> property.  I could easily revert that, but it seemed reasonable to
>> uniformize things in this case.
>
> Thanks, but please find a better name for this property.  Something
> like compilation-meta-data, perhaps?

All right, but since it's just a t-or-nil marking, I don't think
metadata is a very good description.  I considered 'compilation-info'
but that seems related to compiler warnings and errors.

I was looking for something that generally describes headers and
footers, prefaces and epilogues, etc., but I'm missing a good word for
that.

bug#59888: [PATCH] Add 'grep-use-headings'

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: juri@linkov.net,  stefankangas@gmail.com,  59888@debbugs.gnu.org,
>   larsi@gnus.org
> Date: Fri, 09 Dec 2022 21:03:17 +0100
> 
> > Thanks, but please find a better name for this property.  Something
> > like compilation-meta-data, perhaps?
> 
> All right, but since it's just a t-or-nil marking, I don't think
> metadata is a very good description.  I considered 'compilation-info'
> but that seems related to compiler warnings and errors.

How about 'compilation-details'?

> I was looking for something that generally describes headers and
> footers, prefaces and epilogues, etc., but I'm missing a good word for
> that.

If "details" doesn't fit the bill, how about "decorations"?

bug#59888: [PATCH] Add 'grep-use-headings'

[Gregory Heytings]

>
> I was looking for something that generally describes headers and 
> footers, prefaces and epilogues, etc., but I'm missing a good word for 
> that.
>

Decorations?  Frills?

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

> I was looking for something that generally describes headers and
> footers, prefaces and epilogues, etc., but I'm missing a good word for
> that.

This suggests "marginals":
https://english.stackexchange.com/questions/24060/what-word-defines-a-category-suited-for-both-header-and-footer

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

On Fri,  9 Dec 2022 at 22:29, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Cc: juri@linkov.net,  stefankangas@gmail.com,  59888@debbugs.gnu.org,
>>   larsi@gnus.org
>> Date: Fri, 09 Dec 2022 21:03:17 +0100
>> 
>> > Thanks, but please find a better name for this property.  Something
>> > like compilation-meta-data, perhaps?
>> 
>> All right, but since it's just a t-or-nil marking, I don't think
>> metadata is a very good description.  I considered 'compilation-info'
>> but that seems related to compiler warnings and errors.
>
> How about 'compilation-details'?
>
>> I was looking for something that generally describes headers and
>> footers, prefaces and epilogues, etc., but I'm missing a good word for
>> that.
>
> If "details" doesn't fit the bill, how about "decorations"?

Would you be okay with 'compilation-annotation'?  "Decoration" sounds
good, but I like "annotation" slightly better.

Thanks for all the suggestions.

bug#59888: [PATCH] Add 'grep-use-headings'

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: juri@linkov.net,  stefankangas@gmail.com,  59888@debbugs.gnu.org,
>   larsi@gnus.org
> Date: Sat, 10 Dec 2022 21:08:05 +0100
> 
> On Fri,  9 Dec 2022 at 22:29, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel@gmail.com>
> >> Cc: juri@linkov.net,  stefankangas@gmail.com,  59888@debbugs.gnu.org,
> >>   larsi@gnus.org
> >> Date: Fri, 09 Dec 2022 21:03:17 +0100
> >> 
> >> > Thanks, but please find a better name for this property.  Something
> >> > like compilation-meta-data, perhaps?
> >> 
> >> All right, but since it's just a t-or-nil marking, I don't think
> >> metadata is a very good description.  I considered 'compilation-info'
> >> but that seems related to compiler warnings and errors.
> >
> > How about 'compilation-details'?
> >
> >> I was looking for something that generally describes headers and
> >> footers, prefaces and epilogues, etc., but I'm missing a good word for
> >> that.
> >
> > If "details" doesn't fit the bill, how about "decorations"?
> 
> Would you be okay with 'compilation-annotation'?  "Decoration" sounds
> good, but I like "annotation" slightly better.

Could be.  Or how about compilation-aux-data?

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 80 bytes --]

I've attached a new version of the patch incorporating the latest
discussions.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Introduce-compilation-annotation-text-property.patch --]
[-- Type: text/x-patch, Size: 3951 bytes --]

From 8aa2bebbe1b12094963fdeb3251f3ef3c22ea5e9 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Thu, 8 Dec 2022 21:05:10 +0100
Subject: [PATCH 1/2] Introduce 'compilation-annotation' text property

It is meant to mark parts of compilation buffers which do not
correspond to process output.

* lisp/progmodes/compile.el (compilation-insert-annotation): New
function.
(compilation-start, compilation-handle-exit): Use it.
(compilation--ensure-parse) Rely on 'compilation-annotation' property
instead of 'compilation-header-end'.
---
 lisp/progmodes/compile.el | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index e8ada9388e..3aa6867533 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1675,7 +1675,7 @@ compilation--ensure-parse
             (set-marker (make-marker)
                         (save-excursion
                           (goto-char (point-min))
-                          (text-property-search-forward 'compilation-header-end)
+                          (text-property-search-forward 'compilation-annotation)
                           ;; If we have no end marker, this will be
                           ;; `point-min' still.
                           (point)))))
@@ -1823,6 +1823,14 @@ compilation--update-in-progress-mode-line
   ;; buffers when it changes from nil to non-nil or vice-versa.
   (unless compilation-in-progress (force-mode-line-update t)))
 
+(defun compilation-insert-annotation (&rest args)
+  "Insert ARGS at point, adding the `compilation-annotation' text property.
+This property is used to distinguish output of the compilation
+process from additional information inserted by Emacs."
+  (let ((start (point)))
+    (apply #'insert args)
+    (put-text-property start (point) 'compilation-annotation t)))
+
 ;;;###autoload
 (defun compilation-start (command &optional mode name-function highlight-regexp
                                   continue)
@@ -1944,17 +1952,16 @@ compilation-start
             (setq-local compilation-auto-jump-to-next t))
         (when (zerop (buffer-size))
 	  ;; Output a mode setter, for saving and later reloading this buffer.
-	  (insert "-*- mode: " name-of-mode
-		  "; default-directory: "
-                  (prin1-to-string (abbreviate-file-name default-directory))
-		  " -*-\n"))
-        (insert (format "%s started at %s\n\n"
-			mode-name
-			(substring (current-time-string) 0 19))
-		command "\n")
-        ;; Mark the end of the header so that we don't interpret
-        ;; anything in it as an error.
-        (put-text-property (1- (point)) (point) 'compilation-header-end t)
+	  (compilation-insert-annotation
+           "-*- mode: " name-of-mode
+           "; default-directory: "
+           (prin1-to-string (abbreviate-file-name default-directory))
+	   " -*-\n"))
+        (compilation-insert-annotation
+         (format "%s started at %s\n\n"
+                 mode-name
+		 (substring (current-time-string) 0 19))
+	 command "\n")
 	(setq thisdir default-directory))
       (set-buffer-modified-p nil))
     ;; Pop up the compilation buffer.
@@ -2436,13 +2443,13 @@ compilation-handle-exit
 	(cur-buffer (current-buffer)))
     ;; Record where we put the message, so we can ignore it later on.
     (goto-char omax)
-    (insert ?\n mode-name " " (car status))
+    (compilation-insert-annotation ?\n mode-name " " (car status))
     (if (and (numberp compilation-window-height)
 	     (zerop compilation-window-height))
 	(message "%s" (cdr status)))
     (if (bolp)
 	(forward-char -1))
-    (insert " at " (substring (current-time-string) 0 19))
+    (compilation-insert-annotation " at " (substring (current-time-string) 0 19))
     (goto-char (point-max))
     ;; Prevent that message from being recognized as a compilation error.
     (add-text-properties omax (point)
-- 
2.38.1


[-- Attachment #3: 0002-New-user-option-grep-use-headings.patch --]
[-- Type: text/x-patch, Size: 6338 bytes --]

From e1209a93e09792cbb61bde9ae3ac8838c76861a5 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Wed, 7 Dec 2022 18:44:07 +0100
Subject: [PATCH 2/2] New user option 'grep-use-headings'

* lisp/progmodes/grep.el (grep-heading-regexp): New user option.
(grep-heading): New face.
(grep--heading-format, grep--heading-state, grep--heading-filter):
Filter function for grep processes and supporting variables.
(grep-use-headings): New user option.
(grep-mode): Use the above, if applicable.
---
 etc/NEWS                          |  8 ++++
 lisp/progmodes/grep.el            | 69 +++++++++++++++++++++++++++++++
 test/lisp/progmodes/grep-tests.el | 14 +++++++
 3 files changed, 91 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 3eeef0ab4c..ac5ed7cbf3 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -41,6 +41,14 @@ connection.
 \f
 * Changes in Specialized Modes and Packages in Emacs 30.1
 
+** Compile
+
+*** New user option 'grep-use-headings'.
+When non-nil, the output of Grep is split into sections, one for each
+file, instead of having file names prefixed to each line.  It is
+equivalent to the --heading option of some tools such as 'git grep'
+and 'rg'.
+
 ** VC
 
 ---
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 2446e86abb..c6be39fbb7 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -30,6 +30,7 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib))
+(eval-when-compile (require 'rx))
 (require 'compile)
 
 (defgroup grep nil
@@ -457,6 +458,35 @@ grep-search-path
   :type '(repeat (choice (const :tag "Default" nil)
 			 (string :tag "Directory"))))
 
+(defcustom grep-use-headings nil
+  "If non-nil, subdivide grep output into sections, one per file."
+  :type 'boolean
+  :version "30.1")
+
+(defcustom grep-heading-regexp
+  (rx bol
+      (or
+       (group-n 2
+         (group-n 1 (+ (not (any 0 ?\n))))
+         0)
+       (group-n 2
+        (group-n 1 (+? nonl))
+        (any ?: ?- ?=)))
+      (+ digit)
+      (any ?: ?- ?=))
+  "Regexp used to create headings from grep output lines.
+It should be anchored at beginning of line.  The first capture
+group, if present, should match the heading associated to the
+line.  The buffer range of the second capture, if present, is
+made invisible (presumably because displaying it would be
+redundant)."
+  :type 'regexp
+  :version "30.1")
+
+(defface grep-heading `((t :inherit ,grep-hit-face))
+  "Face of headings when using `grep-heading-mode'."
+  :version "30.1")
+
 (defvar grep-find-abbreviate-properties
   (let ((ellipsis (if (char-displayable-p ?…) "[…]" "[...]"))
         (map (make-sparse-keymap)))
@@ -612,6 +642,40 @@ grep-filter
         (while (re-search-forward "\033\\[[0-9;]*[mK]" end 1)
           (replace-match "" t t))))))
 
+(defvar grep--heading-format
+  (eval-when-compile
+    (let ((title (propertize "%s"
+                             'font-lock-face 'grep-heading
+                             'outline-level 1)))
+      (propertize (concat title "\n") 'compilation-annotation t)))
+  "Format string of grep headings.
+This is passed to `format' with one argument, the text of the
+first capture group of `grep-heading-regexp'.")
+
+(defvar-local grep--heading-state nil
+  "Variable to keep track of the `grep--heading-filter' state.")
+
+(defun grep--heading-filter ()
+  "Filter function to add headings to output of a grep process."
+  (unless grep--heading-state
+    (setq grep--heading-state (cons (point-min-marker) nil)))
+  (save-excursion
+    (let ((limit (car grep--heading-state)))
+      ;; Move point to the old limit and update limit marker.
+      (move-marker limit (prog1 (pos-bol) (goto-char limit)))
+      (while (re-search-forward grep-heading-regexp limit t)
+        (unless (get-text-property (point) 'compilation-annotation)
+          (let ((heading (match-string-no-properties 1))
+                (start (match-beginning 2))
+                (end (match-end 2)))
+            (when start
+              (put-text-property start end 'invisible t))
+            (when (and heading (not (equal heading (cdr grep--heading-state))))
+              (save-excursion
+                (forward-line 0)
+                (insert-before-markers (format grep--heading-format heading)))
+              (setf (cdr grep--heading-state) heading))))))))
+
 (defun grep-probe (command args &optional func result)
   (let (process-file-side-effects)
     (equal (condition-case nil
@@ -906,6 +970,11 @@ grep-mode
   (add-function :filter-return (local 'kill-transform-function)
                 (lambda (string)
                   (string-replace "\0" ":" string)))
+  (when grep-use-headings
+    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
+    (setq-local outline-search-function #'outline-search-level
+                outline-level (lambda () (get-text-property
+                                          (point) 'outline-level))))
   (add-hook 'compilation-filter-hook #'grep-filter nil t))
 
 (defun grep--save-buffers ()
diff --git a/test/lisp/progmodes/grep-tests.el b/test/lisp/progmodes/grep-tests.el
index 101052c5ad..51f4606639 100644
--- a/test/lisp/progmodes/grep-tests.el
+++ b/test/lisp/progmodes/grep-tests.el
@@ -66,4 +66,18 @@ grep-tests--rgrep-abbreviate-properties-windows-nt-sh-semantics
     (cl-letf (((symbol-function 'w32-shell-dos-semantics) #'ignore))
       (grep-tests--check-rgrep-abbreviation))))
 
+(ert-deftest grep-tests--grep-heading-regexp-without-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "filename%c123%ctext" sep sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "filename"))
+      (should (equal (match-string 2 string) (format "filename%c" sep))))))
+
+(ert-deftest grep-tests--grep-heading-regexp-with-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "funny:0:filename%c123%ctext" 0 sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "funny:0:filename"))
+      (should (equal (match-string 2 string) "funny:0:filename\0")))))
+
 ;;; grep-tests.el ends here
-- 
2.38.1


[-- Attachment #4: Type: text/plain, Size: 535 bytes --]


There is still an issue I'm aware of: if one saves a grep buffer to a
file and later opens the file, the headings are inserted a second time.
I've tried a bit to display the headings only using text display
properties (instead of inserting the actual text into the buffer), but
wasn't successful so far.  Other, less elegant solutions are possible
(e.g. deleting all headings either when saving or when reading again the
file).

We could install this change now and polish it later or continue this
discussion -- both are fine by me.

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

> There is still an issue I'm aware of: if one saves a grep buffer to a
> file and later opens the file, the headings are inserted a second time.
> I've tried a bit to display the headings only using text display
> properties (instead of inserting the actual text into the buffer), but
> wasn't successful so far.  Other, less elegant solutions are possible
> (e.g. deleting all headings either when saving or when reading again the
> file).

This reminds a similar problem that the file etc/grep.txt
can't be edited in grep-mode because then it will lose
all grep-filtered escape sequences on saving.

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 120 bytes --]

I've been using the grep-use-headings locally for a long time and it
works well.  Should the the patches be installed?


[-- Attachment #2: 0002-New-user-option-grep-use-headings.patch --]
[-- Type: text/x-patch, Size: 6464 bytes --]

From 77ea46f6dc50a8f463bd2f51ce9d0585de0bb55c Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Wed, 7 Dec 2022 18:44:07 +0100
Subject: [PATCH 2/2] New user option 'grep-use-headings'

* lisp/progmodes/grep.el (grep-heading-regexp): New user option.
(grep-heading): New face.
(grep--heading-format, grep--heading-state, grep--heading-filter):
Filter function for grep processes and supporting variables.
(grep-use-headings): New user option.
(grep-mode): Use the above, if applicable.
---
 etc/NEWS                          |  8 ++++
 lisp/progmodes/grep.el            | 69 +++++++++++++++++++++++++++++++
 test/lisp/progmodes/grep-tests.el | 14 +++++++
 3 files changed, 91 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 4b0e4e6bd46..ca2e8011510 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -95,6 +95,14 @@ If you want to get back the old behavior, set the user option to the value
     (setopt gdb-locals-table-row-config
             `((type . 0) (name . 0) (value . ,gdb-locals-value-limit)))
 
+** Compile
+
+*** New user option 'grep-use-headings'.
+When non-nil, the output of Grep is split into sections, one for each
+file, instead of having file names prefixed to each line.  It is
+equivalent to the --heading option of some tools such as 'git grep'
+and 'rg'.
+
 ** VC
 
 ---
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 0da16b44dda..9e3cec89c01 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -30,6 +30,7 @@
 ;;; Code:
 
 (eval-when-compile (require 'cl-lib))
+(eval-when-compile (require 'rx))
 (require 'compile)
 
 (defgroup grep nil
@@ -457,6 +458,35 @@ grep-search-path
   :type '(repeat (choice (const :tag "Default" nil)
 			 (string :tag "Directory"))))
 
+(defcustom grep-use-headings nil
+  "If non-nil, subdivide grep output into sections, one per file."
+  :type 'boolean
+  :version "30.1")
+
+(defcustom grep-heading-regexp
+  (rx bol
+      (or
+       (group-n 2
+         (group-n 1 (+ (not (any 0 ?\n))))
+         0)
+       (group-n 2
+        (group-n 1 (+? nonl))
+        (any ?: ?- ?=)))
+      (+ digit)
+      (any ?: ?- ?=))
+  "Regexp used to create headings from grep output lines.
+It should be anchored at beginning of line.  The first capture
+group, if present, should match the heading associated to the
+line.  The buffer range of the second capture, if present, is
+made invisible (presumably because displaying it would be
+redundant)."
+  :type 'regexp
+  :version "30.1")
+
+(defface grep-heading `((t :inherit ,grep-hit-face))
+  "Face of headings when `grep-use-headings' is non-nil."
+  :version "30.1")
+
 (defvar grep-find-abbreviate-properties
   (let ((ellipsis (if (char-displayable-p ?…) "[…]" "[...]"))
         (map (make-sparse-keymap)))
@@ -612,6 +642,40 @@ grep-filter
         (while (re-search-forward "\033\\[[0-9;]*[mK]" end 1)
           (replace-match "" t t))))))
 
+(defvar grep--heading-format
+  (eval-when-compile
+    (let ((title (propertize "%s"
+                             'font-lock-face 'grep-heading
+                             'outline-level 1)))
+      (propertize (concat title "\n") 'compilation-annotation t)))
+  "Format string of grep headings.
+This is passed to `format' with one argument, the text of the
+first capture group of `grep-heading-regexp'.")
+
+(defvar-local grep--heading-state nil
+  "Variable to keep track of the `grep--heading-filter' state.")
+
+(defun grep--heading-filter ()
+  "Filter function to add headings to output of a grep process."
+  (unless grep--heading-state
+    (setq grep--heading-state (cons (point-min-marker) nil)))
+  (save-excursion
+    (let ((limit (car grep--heading-state)))
+      ;; Move point to the old limit and update limit marker.
+      (move-marker limit (prog1 (pos-bol) (goto-char limit)))
+      (while (re-search-forward grep-heading-regexp limit t)
+        (unless (get-text-property (point) 'compilation-annotation)
+          (let ((heading (match-string-no-properties 1))
+                (start (match-beginning 2))
+                (end (match-end 2)))
+            (when start
+              (put-text-property start end 'invisible t))
+            (when (and heading (not (equal heading (cdr grep--heading-state))))
+              (save-excursion
+                (forward-line 0)
+                (insert-before-markers (format grep--heading-format heading)))
+              (setf (cdr grep--heading-state) heading))))))))
+
 (defun grep-probe (command args &optional func result)
   (let (process-file-side-effects)
     (equal (condition-case nil
@@ -906,6 +970,11 @@ grep-mode
   (add-function :filter-return (local 'kill-transform-function)
                 (lambda (string)
                   (string-replace "\0" ":" string)))
+  (when grep-use-headings
+    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
+    (setq-local outline-search-function #'outline-search-level
+                outline-level (lambda () (get-text-property
+                                          (point) 'outline-level))))
   (add-hook 'compilation-filter-hook #'grep-filter nil t))
 
 (defun grep--save-buffers ()
diff --git a/test/lisp/progmodes/grep-tests.el b/test/lisp/progmodes/grep-tests.el
index 39307999d6d..9b7f83086bf 100644
--- a/test/lisp/progmodes/grep-tests.el
+++ b/test/lisp/progmodes/grep-tests.el
@@ -66,4 +66,18 @@ grep-tests--rgrep-abbreviate-properties-windows-nt-sh-semantics
     (cl-letf (((symbol-function 'w32-shell-dos-semantics) #'ignore))
       (grep-tests--check-rgrep-abbreviation))))
 
+(ert-deftest grep-tests--grep-heading-regexp-without-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "filename%c123%ctext" sep sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "filename"))
+      (should (equal (match-string 2 string) (format "filename%c" sep))))))
+
+(ert-deftest grep-tests--grep-heading-regexp-with-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "funny:0:filename%c123%ctext" 0 sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "funny:0:filename"))
+      (should (equal (match-string 2 string) "funny:0:filename\0")))))
+
 ;;; grep-tests.el ends here
-- 
2.39.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Introduce-compilation-annotation-text-property.patch --]
[-- Type: text/x-patch, Size: 3953 bytes --]

From 6d97ae3b43fe893f2ec7dab67c14f06e1eb9a3fb Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Thu, 8 Dec 2022 21:05:10 +0100
Subject: [PATCH 1/2] Introduce 'compilation-annotation' text property

It is meant to mark parts of compilation buffers which do not
correspond to process output.

* lisp/progmodes/compile.el (compilation-insert-annotation): New
function.
(compilation-start, compilation-handle-exit): Use it.
(compilation--ensure-parse) Rely on 'compilation-annotation' property
instead of 'compilation-header-end'.
---
 lisp/progmodes/compile.el | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index ccf64fb670b..6d151db8a83 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1706,7 +1706,7 @@ compilation--ensure-parse
             (set-marker (make-marker)
                         (save-excursion
                           (goto-char (point-min))
-                          (text-property-search-forward 'compilation-header-end)
+                          (text-property-search-forward 'compilation-annotation)
                           ;; If we have no end marker, this will be
                           ;; `point-min' still.
                           (point)))))
@@ -1854,6 +1854,14 @@ compilation--update-in-progress-mode-line
   ;; buffers when it changes from nil to non-nil or vice-versa.
   (unless compilation-in-progress (force-mode-line-update t)))
 
+(defun compilation-insert-annotation (&rest args)
+  "Insert ARGS at point, adding the `compilation-annotation' text property.
+This property is used to distinguish output of the compilation
+process from additional information inserted by Emacs."
+  (let ((start (point)))
+    (apply #'insert args)
+    (put-text-property start (point) 'compilation-annotation t)))
+
 ;;;###autoload
 (defun compilation-start (command &optional mode name-function highlight-regexp
                                   continue)
@@ -1975,17 +1983,16 @@ compilation-start
             (setq-local compilation-auto-jump-to-next t))
         (when (zerop (buffer-size))
 	  ;; Output a mode setter, for saving and later reloading this buffer.
-	  (insert "-*- mode: " name-of-mode
-		  "; default-directory: "
-                  (prin1-to-string (abbreviate-file-name default-directory))
-		  " -*-\n"))
-        (insert (format "%s started at %s\n\n"
-			mode-name
-			(substring (current-time-string) 0 19))
-		command "\n")
-        ;; Mark the end of the header so that we don't interpret
-        ;; anything in it as an error.
-        (put-text-property (1- (point)) (point) 'compilation-header-end t)
+	  (compilation-insert-annotation
+           "-*- mode: " name-of-mode
+           "; default-directory: "
+           (prin1-to-string (abbreviate-file-name default-directory))
+	   " -*-\n"))
+        (compilation-insert-annotation
+         (format "%s started at %s\n\n"
+                 mode-name
+		 (substring (current-time-string) 0 19))
+	 command "\n")
 	(setq thisdir default-directory))
       (set-buffer-modified-p nil))
     ;; Pop up the compilation buffer.
@@ -2467,13 +2474,13 @@ compilation-handle-exit
 	(cur-buffer (current-buffer)))
     ;; Record where we put the message, so we can ignore it later on.
     (goto-char omax)
-    (insert ?\n mode-name " " (car status))
+    (compilation-insert-annotation ?\n mode-name " " (car status))
     (if (and (numberp compilation-window-height)
 	     (zerop compilation-window-height))
 	(message "%s" (cdr status)))
     (if (bolp)
 	(forward-char -1))
-    (insert " at " (substring (current-time-string) 0 19))
+    (compilation-insert-annotation " at " (substring (current-time-string) 0 19))
     (goto-char (point-max))
     ;; Prevent that message from being recognized as a compilation error.
     (add-text-properties omax (point)
-- 
2.39.2

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

> I've been using the grep-use-headings locally for a long time and it
> works well.  Should the the patches be installed?

Thanks, I'd like to try out your patches for a few days.

bug#59888: [PATCH] Add 'grep-use-headings'

[Robert Pluim]

>>>>> On Sat, 25 Feb 2023 09:34:56 +0100, Augusto Stoffel <arstoffel@gmail.com> said:

    Augusto> I've been using the grep-use-headings locally for a long time and it
    Augusto> works well.  Should the the patches be installed?

    Augusto> From 77ea46f6dc50a8f463bd2f51ce9d0585de0bb55c Mon Sep 17 00:00:00 2001
    Augusto> From: Augusto Stoffel <arstoffel@gmail.com>
    Augusto> Date: Wed, 7 Dec 2022 18:44:07 +0100
    Augusto> Subject: [PATCH 2/2] New user option 'grep-use-headings'

    Augusto> * lisp/progmodes/grep.el (grep-heading-regexp): New user option.
    Augusto> (grep-heading): New face.
    Augusto> (grep--heading-format, grep--heading-state, grep--heading-filter):
    Augusto> Filter function for grep processes and supporting variables.
    Augusto> (grep-use-headings): New user option.
    Augusto> (grep-mode): Use the above, if applicable.
    Augusto> ---
    Augusto>  etc/NEWS                          |  8 ++++
    Augusto>  lisp/progmodes/grep.el            | 69 +++++++++++++++++++++++++++++++
    Augusto>  test/lisp/progmodes/grep-tests.el | 14 +++++++
    Augusto>  3 files changed, 91 insertions(+)

    Augusto> diff --git a/etc/NEWS b/etc/NEWS
    Augusto> index 4b0e4e6bd46..ca2e8011510 100644
    Augusto> --- a/etc/NEWS
    Augusto> +++ b/etc/NEWS
    Augusto> @@ -95,6 +95,14 @@ If you want to get back the old behavior, set the user option to the value
    Augusto>      (setopt gdb-locals-table-row-config
    Augusto>              `((type . 0) (name . 0) (value . ,gdb-locals-value-limit)))
 
    Augusto> +** Compile
    Augusto> +
    Augusto> +*** New user option 'grep-use-headings'.
    Augusto> +When non-nil, the output of Grep is split into sections, one for each
    Augusto> +file, instead of having file names prefixed to each line.  It is
    Augusto> +equivalent to the --heading option of some tools such as 'git grep'
    Augusto> +and 'rg'.
    Augusto> +

You also add a face and a user option for the regexp, they should be
mentioned in NEWS (and the manual, if relevant)
 
    Augusto> +(defvar grep--heading-format
    Augusto> +  (eval-when-compile
    Augusto> +    (let ((title (propertize "%s"
    Augusto> +                             'font-lock-face 'grep-heading
    Augusto> +                             'outline-level 1)))
    Augusto> +      (propertize (concat title "\n") 'compilation-annotation t)))
    Augusto> +  "Format string of grep headings.
    Augusto> +This is passed to `format' with one argument, the text of the
    Augusto> +first capture group of `grep-heading-regexp'.")
    Augusto> +
    Augusto> +(defvar-local grep--heading-state nil
    Augusto> +  "Variable to keep track of the `grep--heading-filter' state.")
    Augusto> +
    Augusto> +(defun grep--heading-filter ()
    Augusto> +  "Filter function to add headings to output of a grep process."
    Augusto> +  (unless grep--heading-state
    Augusto> +    (setq grep--heading-state (cons (point-min-marker) nil)))
    Augusto> +  (save-excursion
    Augusto> +    (let ((limit (car grep--heading-state)))
    Augusto> +      ;; Move point to the old limit and update limit marker.
    Augusto> +      (move-marker limit (prog1 (pos-bol) (goto-char limit)))
    Augusto> +      (while (re-search-forward grep-heading-regexp limit t)
    Augusto> +        (unless (get-text-property (point) 'compilation-annotation)
    Augusto> +          (let ((heading (match-string-no-properties 1))
    Augusto> +                (start (match-beginning 2))
    Augusto> +                (end (match-end 2)))
    Augusto> +            (when start
    Augusto> +              (put-text-property start end 'invisible t))
    Augusto> +            (when (and heading (not (equal heading (cdr grep--heading-state))))
    Augusto> +              (save-excursion
    Augusto> +                (forward-line 0)

Thatʼs the same as (goto-char (pos-bol)) because of a wrinkle in the
implementation of `forward-line'.  It might even be faster, but Iʼve
not measured it :-)

Robert
-- 

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

On Sun, 26 Feb 2023 at 14:17, Robert Pluim wrote:

> You also add a face and a user option for the regexp, they should be
> mentioned in NEWS (and the manual, if relevant)

Thanks, I added the face to the NEWS (but not the regexp, which is not a
user option although it can be modified if you are desperate).

>     Augusto> +                (forward-line 0)
>
> Thatʼs the same as (goto-char (pos-bol)) because of a wrinkle in the
> implementation of `forward-line'.  It might even be faster, but Iʼve
> not measured it :-)

Ugh, that was an anachronism.  It will be fixed when I send the final
patch.

bug#59888: [PATCH] Add 'grep-use-headings'

[Robert Pluim]

>>>>> On Sun, 26 Feb 2023 16:07:13 +0100, Augusto Stoffel <arstoffel@gmail.com> said:

    Augusto> On Sun, 26 Feb 2023 at 14:17, Robert Pluim wrote:
    >> You also add a face and a user option for the regexp, they should be
    >> mentioned in NEWS (and the manual, if relevant)

    Augusto> Thanks, I added the face to the NEWS (but not the regexp, which is not a
    Augusto> user option although it can be modified if you are desperate).

Itʼs a `defcustom', so by definition itʼs a user option. If youʼre not
expecting it to be modified, maybe make it a `defvar'.

Robert
-- 

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

On Mon, 27 Feb 2023 at 07:24, Robert Pluim wrote:

> Itʼs a `defcustom', so by definition itʼs a user option. If youʼre not
> expecting it to be modified, maybe make it a `defvar'.

Oops, I wrote this a couple months back and had forgotten.

I don't know if you have an opinion.  `grep-regexp-alist' is a defconst,
which seems exaggerated, but `grep-match-regexp' is a defcustom although
nobody should want to change this except to work around bugs.

My new variable has a similar "degree of technicality" and I think that
asks for be a defvar.

bug#59888: [PATCH] Add 'grep-heading-mode'

[Mattias Engdegård]

> +(eval-when-compile (require 'rx))

Is this needed? The `rx` macro is autoloaded.

bug#59888: [PATCH] Add 'grep-use-headings'

[Robert Pluim]

>>>>> On Mon, 27 Feb 2023 12:26:08 +0100, Augusto Stoffel <arstoffel@gmail.com> said:

    Augusto> On Mon, 27 Feb 2023 at 07:24, Robert Pluim wrote:
    >> Itʼs a `defcustom', so by definition itʼs a user option. If youʼre not
    >> expecting it to be modified, maybe make it a `defvar'.

    Augusto> Oops, I wrote this a couple months back and had forgotten.

    Augusto> I don't know if you have an opinion.  `grep-regexp-alist' is a defconst,
    Augusto> which seems exaggerated, but `grep-match-regexp' is a defcustom although
    Augusto> nobody should want to change this except to work around bugs.

    Augusto> My new variable has a similar "degree of technicality" and I think that
    Augusto> asks for be a defvar.

I think I agree.

Robert
-- 

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

>> Thatʼs the same as (goto-char (pos-bol)) because of a wrinkle in the
>> implementation of `forward-line'.  It might even be faster, but Iʼve
>> not measured it :-)
>
> Ugh, that was an anachronism.  It will be fixed when I send the final
> patch.

I tried out your patch, and everything works nicely.
So I guess it could be pushed when you send the final version.

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 189 bytes --]

On Mon, 27 Feb 2023 at 20:53, Juri Linkov wrote:

> I tried out your patch, and everything works nicely.
> So I guess it could be pushed when you send the final version.

There you go :-)


[-- Attachment #2: 0002-New-user-option-grep-use-headings.patch --]
[-- Type: text/x-patch, Size: 6339 bytes --]

From d141b5c31718bf312cd06cd85c2865621753ad7b Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Wed, 7 Dec 2022 18:44:07 +0100
Subject: [PATCH 2/2] New user option 'grep-use-headings'

* lisp/progmodes/grep.el (grep-heading-regexp): New user option.
(grep-heading): New face.
(grep--heading-format, grep--heading-state, grep--heading-filter):
Filter function for grep processes and supporting variables.
(grep-use-headings): New user option.
(grep-mode): Use the above, if applicable.
---
 etc/NEWS                          |  9 +++++
 lisp/progmodes/grep.el            | 66 +++++++++++++++++++++++++++++++
 test/lisp/progmodes/grep-tests.el | 14 +++++++
 3 files changed, 89 insertions(+)

diff --git a/etc/NEWS b/etc/NEWS
index 4b0e4e6bd46..58bbd083ef8 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -95,6 +95,15 @@ If you want to get back the old behavior, set the user option to the value
     (setopt gdb-locals-table-row-config
             `((type . 0) (name . 0) (value . ,gdb-locals-value-limit)))
 
+** Compile
+
+*** New user option 'grep-use-headings'.
+When non-nil, the output of Grep is split into sections, one for each
+file, instead of having file names prefixed to each line.  It is
+equivalent to the --heading option of some tools such as 'git grep'
+and 'rg'.  The headings are displayed using the new 'grep-heading'
+face.
+
 ** VC
 
 ---
diff --git a/lisp/progmodes/grep.el b/lisp/progmodes/grep.el
index 0da16b44dda..82e9c5d8edf 100644
--- a/lisp/progmodes/grep.el
+++ b/lisp/progmodes/grep.el
@@ -457,6 +457,33 @@ grep-search-path
   :type '(repeat (choice (const :tag "Default" nil)
 			 (string :tag "Directory"))))
 
+(defcustom grep-use-headings nil
+  "If non-nil, subdivide grep output into sections, one per file."
+  :type 'boolean
+  :version "30.1")
+
+(defface grep-heading `((t :inherit ,grep-hit-face))
+  "Face of headings when `grep-use-headings' is non-nil."
+  :version "30.1")
+
+(defvar grep-heading-regexp
+  (rx bol
+      (or
+       (group-n 2
+         (group-n 1 (+ (not (any 0 ?\n))))
+         0)
+       (group-n 2
+        (group-n 1 (+? nonl))
+        (any ?: ?- ?=)))
+      (+ digit)
+      (any ?: ?- ?=))
+  "Regexp used to create headings from grep output lines.
+It should be anchored at beginning of line.  The first capture
+group, if present, should match the heading associated to the
+line.  The buffer range of the second capture, if present, is
+made invisible (presumably because displaying it would be
+redundant).")
+
 (defvar grep-find-abbreviate-properties
   (let ((ellipsis (if (char-displayable-p ?…) "[…]" "[...]"))
         (map (make-sparse-keymap)))
@@ -612,6 +639,40 @@ grep-filter
         (while (re-search-forward "\033\\[[0-9;]*[mK]" end 1)
           (replace-match "" t t))))))
 
+(defvar grep--heading-format
+  (eval-when-compile
+    (let ((title (propertize "%s"
+                             'font-lock-face 'grep-heading
+                             'outline-level 1)))
+      (propertize (concat title "\n") 'compilation-annotation t)))
+  "Format string of grep headings.
+This is passed to `format' with one argument, the text of the
+first capture group of `grep-heading-regexp'.")
+
+(defvar-local grep--heading-state nil
+  "Variable to keep track of the `grep--heading-filter' state.")
+
+(defun grep--heading-filter ()
+  "Filter function to add headings to output of a grep process."
+  (unless grep--heading-state
+    (setq grep--heading-state (cons (point-min-marker) nil)))
+  (save-excursion
+    (let ((limit (car grep--heading-state)))
+      ;; Move point to the old limit and update limit marker.
+      (move-marker limit (prog1 (pos-bol) (goto-char limit)))
+      (while (re-search-forward grep-heading-regexp limit t)
+        (unless (get-text-property (point) 'compilation-annotation)
+          (let ((heading (match-string-no-properties 1))
+                (start (match-beginning 2))
+                (end (match-end 2)))
+            (when start
+              (put-text-property start end 'invisible t))
+            (when (and heading (not (equal heading (cdr grep--heading-state))))
+              (save-excursion
+                (goto-char (pos-bol))
+                (insert-before-markers (format grep--heading-format heading)))
+              (setf (cdr grep--heading-state) heading))))))))
+
 (defun grep-probe (command args &optional func result)
   (let (process-file-side-effects)
     (equal (condition-case nil
@@ -906,6 +967,11 @@ grep-mode
   (add-function :filter-return (local 'kill-transform-function)
                 (lambda (string)
                   (string-replace "\0" ":" string)))
+  (when grep-use-headings
+    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
+    (setq-local outline-search-function #'outline-search-level
+                outline-level (lambda () (get-text-property
+                                          (point) 'outline-level))))
   (add-hook 'compilation-filter-hook #'grep-filter nil t))
 
 (defun grep--save-buffers ()
diff --git a/test/lisp/progmodes/grep-tests.el b/test/lisp/progmodes/grep-tests.el
index 39307999d6d..9b7f83086bf 100644
--- a/test/lisp/progmodes/grep-tests.el
+++ b/test/lisp/progmodes/grep-tests.el
@@ -66,4 +66,18 @@ grep-tests--rgrep-abbreviate-properties-windows-nt-sh-semantics
     (cl-letf (((symbol-function 'w32-shell-dos-semantics) #'ignore))
       (grep-tests--check-rgrep-abbreviation))))
 
+(ert-deftest grep-tests--grep-heading-regexp-without-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "filename%c123%ctext" sep sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "filename"))
+      (should (equal (match-string 2 string) (format "filename%c" sep))))))
+
+(ert-deftest grep-tests--grep-heading-regexp-with-null ()
+  (dolist (sep '(?: ?- ?=))
+    (let ((string (format "funny:0:filename%c123%ctext" 0 sep)))
+      (should (string-match grep-heading-regexp string))
+      (should (equal (match-string 1 string) "funny:0:filename"))
+      (should (equal (match-string 2 string) "funny:0:filename\0")))))
+
 ;;; grep-tests.el ends here
-- 
2.39.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-Introduce-compilation-annotation-text-property.patch --]
[-- Type: text/x-patch, Size: 3953 bytes --]

From 8660f5cc050ecf3789574cfc11544b5950b8e949 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Thu, 8 Dec 2022 21:05:10 +0100
Subject: [PATCH 1/2] Introduce 'compilation-annotation' text property

It is meant to mark parts of compilation buffers which do not
correspond to process output.

* lisp/progmodes/compile.el (compilation-insert-annotation): New
function.
(compilation-start, compilation-handle-exit): Use it.
(compilation--ensure-parse) Rely on 'compilation-annotation' property
instead of 'compilation-header-end'.
---
 lisp/progmodes/compile.el | 35 +++++++++++++++++++++--------------
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index ccf64fb670b..6d151db8a83 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1706,7 +1706,7 @@ compilation--ensure-parse
             (set-marker (make-marker)
                         (save-excursion
                           (goto-char (point-min))
-                          (text-property-search-forward 'compilation-header-end)
+                          (text-property-search-forward 'compilation-annotation)
                           ;; If we have no end marker, this will be
                           ;; `point-min' still.
                           (point)))))
@@ -1854,6 +1854,14 @@ compilation--update-in-progress-mode-line
   ;; buffers when it changes from nil to non-nil or vice-versa.
   (unless compilation-in-progress (force-mode-line-update t)))
 
+(defun compilation-insert-annotation (&rest args)
+  "Insert ARGS at point, adding the `compilation-annotation' text property.
+This property is used to distinguish output of the compilation
+process from additional information inserted by Emacs."
+  (let ((start (point)))
+    (apply #'insert args)
+    (put-text-property start (point) 'compilation-annotation t)))
+
 ;;;###autoload
 (defun compilation-start (command &optional mode name-function highlight-regexp
                                   continue)
@@ -1975,17 +1983,16 @@ compilation-start
             (setq-local compilation-auto-jump-to-next t))
         (when (zerop (buffer-size))
 	  ;; Output a mode setter, for saving and later reloading this buffer.
-	  (insert "-*- mode: " name-of-mode
-		  "; default-directory: "
-                  (prin1-to-string (abbreviate-file-name default-directory))
-		  " -*-\n"))
-        (insert (format "%s started at %s\n\n"
-			mode-name
-			(substring (current-time-string) 0 19))
-		command "\n")
-        ;; Mark the end of the header so that we don't interpret
-        ;; anything in it as an error.
-        (put-text-property (1- (point)) (point) 'compilation-header-end t)
+	  (compilation-insert-annotation
+           "-*- mode: " name-of-mode
+           "; default-directory: "
+           (prin1-to-string (abbreviate-file-name default-directory))
+	   " -*-\n"))
+        (compilation-insert-annotation
+         (format "%s started at %s\n\n"
+                 mode-name
+		 (substring (current-time-string) 0 19))
+	 command "\n")
 	(setq thisdir default-directory))
       (set-buffer-modified-p nil))
     ;; Pop up the compilation buffer.
@@ -2467,13 +2474,13 @@ compilation-handle-exit
 	(cur-buffer (current-buffer)))
     ;; Record where we put the message, so we can ignore it later on.
     (goto-char omax)
-    (insert ?\n mode-name " " (car status))
+    (compilation-insert-annotation ?\n mode-name " " (car status))
     (if (and (numberp compilation-window-height)
 	     (zerop compilation-window-height))
 	(message "%s" (cdr status)))
     (if (bolp)
 	(forward-char -1))
-    (insert " at " (substring (current-time-string) 0 19))
+    (compilation-insert-annotation " at " (substring (current-time-string) 0 19))
     (goto-char (point-max))
     ;; Prevent that message from being recognized as a compilation error.
     (add-text-properties omax (point)
-- 
2.39.2

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

close 59888 30.0.50
thanks

>> I tried out your patch, and everything works nicely.
>> So I guess it could be pushed when you send the final version.
>
> There you go :-)

Thanks for this useful feature.  Now pushed to master.
(And immediately customized it to t ;-)

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

> +(defvar grep-heading-regexp
> +  (rx bol
> +      (or
> +       (group-n 2
> +         (group-n 1 (+ (not (any 0 ?\n))))
> +         0)
> +       (group-n 2
> +        (group-n 1 (+? nonl))
> +        (any ?: ?- ?=)))
> +      (+ digit)
> +      (any ?: ?- ?=))

I wonder what is the reason for this regexp to be different
from grep-regexp-alist?  Especially with such additional characters
as ?- and ?=.  This mismatch causes wrong handling of files when
file names contain these characters.

bug#59888: [PATCH] Add 'grep-use-headings'

[Augusto Stoffel]

On Tue, 28 Feb 2023 at 19:24, Juri Linkov wrote:

>> +(defvar grep-heading-regexp
>> +  (rx bol
>> +      (or
>> +       (group-n 2
>> +         (group-n 1 (+ (not (any 0 ?\n))))
>> +         0)
>> +       (group-n 2
>> +        (group-n 1 (+? nonl))
>> +        (any ?: ?- ?=)))
>> +      (+ digit)
>> +      (any ?: ?- ?=))
>
> I wonder what is the reason for this regexp to be different
> from grep-regexp-alist?  Especially with such additional characters
> as ?- and ?=.  This mismatch causes wrong handling of files when
> file names contain these characters.

This is because of context lines:

--8<---------------cut here---------------start------------->8---
$ git grep -nC 1 grep-use-headings
etc/NEWS-99-
etc/NEWS:100:*** New user option 'grep-use-headings'.
etc/NEWS-101-When non-nil, the output of Grep is split into sections, one for each
--
lisp/progmodes/grep.el-459-
lisp/progmodes/grep.el:460:(defcustom grep-use-headings nil
lisp/progmodes/grep.el-461-  "If non-nil, subdivide grep output into sections, one per file."
--
lisp/progmodes/grep.el-465-(defface grep-heading `((t :inherit ,grep-hit-face))
lisp/progmodes/grep.el:466:  "Face of headings when `grep-use-headings' is non-nil."
lisp/progmodes/grep.el-467-  :version "30.1")
--
lisp/progmodes/grep.el-969-                  (string-replace "\0" ":" string)))
lisp/progmodes/grep.el:970:  (when grep-use-headings
lisp/progmodes/grep.el-971-    (add-hook 'compilation-filter-hook #'grep--heading-filter 80 t)
--8<---------------cut here---------------end--------------->8---

(I forget which program uses the =LINE= syntax)

At least when using grep --null, I think there is no confusion possible.  I
wrote a unit test that purports to prove that.

bug#59888: [PATCH] Add 'grep-use-headings'

[Juri Linkov]

>>> +      (any ?: ?- ?=))
>>
>> I wonder what is the reason for this regexp to be different
>> from grep-regexp-alist?  Especially with such additional characters
>> as ?- and ?=.  This mismatch causes wrong handling of files when
>> file names contain these characters.
>
> This is because of context lines:
>
> $ git grep -nC 1 grep-use-headings
> lisp/progmodes/grep.el-459-
> lisp/progmodes/grep.el:460:(defcustom grep-use-headings nil
> lisp/progmodes/grep.el-461-  "If non-nil, subdivide grep output into sections, one per file."
> ...
> At least when using grep --null, I think there is no confusion possible.  I
> wrote a unit test that purports to prove that.

Thanks for mentioning --null, I completely forgot about it.
I've customized 'grep-find-template' to "rg --no-heading"
because ripgrep is fast, but since its headings are not
supported by grep.el, so also needed "--no-heading".
And now after adding "--null" to "rg --no-heading --null"
'grep-use-headings' correctly handles context lines
and file names with numbers separated by dashes.