bug#50207: 28.0.50; ansi-color-compilation-filter and rgrep

[Jim Porter <jporterbugs@gmail.com>]

On 8/25/2021 10:57 PM, Manuel Uberti wrote:
>  From 'emacs -Q', this is what I did:
> 
> - (add-hook 'compilation-filter-hook #'ansi-color-compilation-filter)
> - M-x rgrep RET
> - setq RET
> - RET
> - ~/emacs RET (this is where I keep the entire Emacs source code)
> 
> As you can see from the attached image, some of the results are red and 
> some are not. This is not happening without ansi-color-compilation-filter.y

I encountered this a bit ago, and did a bit of diagnosis, but it ended 
up on my back-burner. I think the issue is due to how the 
compilation-filter-hooks for grep and ansi-color interact. `grep-filter' 
is fairly simple and wants to see both the start and end of an 
ANSI-colorized region, so it "rewinds" to the beginning of a line every 
time it's called. `ansi-color-compilation-filter', on the other hand, is 
smart enough to handle the case where it only sees the start of a 
colorized region in one call, and the end in the next call (see 
`ansi-color-context' for details).

These interact poorly, since normally `grep-filter' reads all the ANSI 
escapes, handles the ones it recognizes, and strips the rest, leaving 
`ansi-color-compilation-filter' with nothing to do. However, when grep's 
output currently shows the start of a colorized region but not the end, 
`grep-filter' doesn't touch that, assuming it can come back to it in the 
next call. By then `ansi-color-compilation-filter' has "stolen" that 
ANSI escape, confusing `grep-filter'.

To solve this, I just turn off `ansi-color-compilation-filter' for grep. 
When things are working right, it should be a no-op, and when things are 
working wrong, you see the behavior described in this bug. It might be 
nice to turn this off automatically though. Right now it's just a sharp 
corner people can cut themselves on.