[Lute.Kamstra.lists@xs4all.nl: Re: junk in *grep* buffers]

[Lute.Kamstra.lists@xs4all.nl: Re: junk in *grep* buffers]

[Richard Stallman]

Would someone please investigate this bug?
Please respond to this message if you investigate it.

------- Start of forwarded message -------
To: Dave Love <fx@gnu.org>
In-Reply-To: <rzqd5t8g8x2.fsf@dpcsport.dl.ac.uk> (Dave Love's message of "Tue,
	05 Apr 2005 23:23:21 +0100")
From: Lute Kamstra <Lute.Kamstra.lists@xs4all.nl>
Date: Wed, 06 Apr 2005 01:46:34 +0200
Cc: emacs-pretest-bug@gnu.org
Subject: Re: junk in *grep* buffers
X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.63

Dave Love <fx@gnu.org> writes:

> Escape sequences appear randomly in *grep* buffers, at least on
> lines with two matches.  You can get different results from running
> the same grep multiple times, but not reproducibly, and you may not
> see the escapes the first time.

[...]

I've noticed a problem that's likely related.

When I do M-x grep and use

  grep -nH -e "(define-minor-mode" lisp/*.el

I get a *grep* buffer with occurrences of "(define-minor-mode" in lisp
files.  Every time that I tried, most lines in this buffer give the
text "(define-minor-mode" the grep-match-face, but a few lines don't
fontify "(define-minor-mode".  The strange thing is that the lines
that don't fontify "(define-minor-mode" are different every time I
invoke grep.

I've got a sneaky suspicion that something goes awry with the
interaction between font-locking and the code in
grep-mode-font-lock-keywords that removes the escape sequences.

Lute.

_______________________________________________
Emacs-pretest-bug mailing list
Emacs-pretest-bug@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-pretest-bug
------- End of forwarded message -------

[Lute.Kamstra.lists@xs4all.nl: Re: junk in *grep* buffers]

[Richard Stallman]

Would someone please investigate this bug?
Please respond to this message if you investigate it.

------- Start of forwarded message -------
To: Dave Love <fx@gnu.org>
In-Reply-To: <rzqd5t8g8x2.fsf@dpcsport.dl.ac.uk> (Dave Love's message of "Tue,
	05 Apr 2005 23:23:21 +0100")
From: Lute Kamstra <Lute.Kamstra.lists@xs4all.nl>
Date: Wed, 06 Apr 2005 01:46:34 +0200
Cc: emacs-pretest-bug@gnu.org
Subject: Re: junk in *grep* buffers
X-Spam-Status: No, hits=0.0 required=5.0 tests=none autolearn=no version=2.63

Dave Love <fx@gnu.org> writes:

> Escape sequences appear randomly in *grep* buffers, at least on
> lines with two matches.  You can get different results from running
> the same grep multiple times, but not reproducibly, and you may not
> see the escapes the first time.

[...]

I've noticed a problem that's likely related.

When I do M-x grep and use

  grep -nH -e "(define-minor-mode" lisp/*.el

I get a *grep* buffer with occurrences of "(define-minor-mode" in lisp
files.  Every time that I tried, most lines in this buffer give the
text "(define-minor-mode" the grep-match-face, but a few lines don't
fontify "(define-minor-mode".  The strange thing is that the lines
that don't fontify "(define-minor-mode" are different every time I
invoke grep.

I've got a sneaky suspicion that something goes awry with the
interaction between font-locking and the code in
grep-mode-font-lock-keywords that removes the escape sequences.

Lute.

_______________________________________________
Emacs-pretest-bug mailing list
Emacs-pretest-bug@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-pretest-bug
------- End of forwarded message -------

Re: Fwd: Re: junk in *grep* buffers

[Stefan Monnier]

> Would someone please investigate this bug?
> Please respond to this message if you investigate it.

I can't investigate it (no time, and no coloring-grep to try it on),
but here is a suggestion.

> When I do M-x grep and use

>   grep -nH -e "(define-minor-mode" lisp/*.el

> I get a *grep* buffer with occurrences of "(define-minor-mode" in lisp
> files.  Every time that I tried, most lines in this buffer give the
> text "(define-minor-mode" the grep-match-face, but a few lines don't
> fontify "(define-minor-mode".  The strange thing is that the lines
> that don't fontify "(define-minor-mode" are different every time I
> invoke grep.

The problem, most likely is the following:

1 - grep sends a partial line like

     foo:123:toto \033[01;41mMATCH\033[00m

2 - font-lock fontifies this, which adds a face property and removes
    the markers, so the text is now:

     foo:123:toto MATCH

3 - grep sends the rest of the line

     bar baz\n

    so the complete line is now

     foo:123:toto MATCH bar baz\n

4 - font-lock is triggered again to fontify the added text, but it works
    a line-at-a-time so it re-fontifies the whole line, what begins by
    removing the `face' property and never re-adds it since the merkers are
    now lost.

So the patch below should fix the problem because it uses the font-lock-face
property which is not cleared by font-lock.

If you find the patch works, please just install it for me,


        Stefan


Index: grep.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/progmodes/grep.el,v
retrieving revision 1.34
diff -u -u -b -r1.34 grep.el
--- grep.el	9 Feb 2005 15:50:36 -0000	1.34
+++ grep.el	26 Apr 2005 21:17:45 -0000
@@ -1,7 +1,7 @@
 ;;; grep.el --- run compiler as inferior of Emacs, parse error messages
 
 ;; Copyright (C) 1985, 1986, 1987, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
-;;   2001, 2002, 2004  Free Software Foundation, Inc.
+;;   2001, 2002, 2004, 2005  Free Software Foundation, Inc.
 
 ;; Author: Roland McGrath <roland@gnu.org>
 ;; Maintainer: FSF
@@ -294,7 +294,7 @@
       (2 compilation-line-face))
      ;; Highlight grep matches and delete markers
      ("\\(\033\\[01;41m\\)\\(.*?\\)\\(\033\\[00m\\(\033\\[K\\)?\\)"
-      (2 grep-match-face)
+      (2 (list 'face nil 'font-lock-face grep-match-face))
       ((lambda (p))
        (progn
 	 ;; Delete markers with `replace-match' because it updates

Re: Fwd: Re: junk in *grep* buffers

[Lute Kamstra]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> When I do M-x grep and use
>>
>>   grep -nH -e "(define-minor-mode" lisp/*.el
>>
>> I get a *grep* buffer with occurrences of "(define-minor-mode" in lisp
>> files.  Every time that I tried, most lines in this buffer give the
>> text "(define-minor-mode" the grep-match-face, but a few lines don't
>> fontify "(define-minor-mode".  The strange thing is that the lines
>> that don't fontify "(define-minor-mode" are different every time I
>> invoke grep.
>
> The problem, most likely is the following:
>
> 1 - grep sends a partial line like
>
>      foo:123:toto \033[01;41mMATCH\033[00m
>
> 2 - font-lock fontifies this, which adds a face property and removes
>     the markers, so the text is now:
>
>      foo:123:toto MATCH
>
> 3 - grep sends the rest of the line
>
>      bar baz\n
>
>     so the complete line is now
>
>      foo:123:toto MATCH bar baz\n
>
> 4 - font-lock is triggered again to fontify the added text, but it works
>     a line-at-a-time so it re-fontifies the whole line, what begins by
>     removing the `face' property and never re-adds it since the merkers are
>     now lost.
>
> So the patch below should fix the problem because it uses the font-lock-face
> property which is not cleared by font-lock.
>
> If you find the patch works, please just install it for me,

Your patch fixes the problem I described.  I'll add a comment
explaining the need to use font-lock-face and commit it.

Lute.

Re: Fwd: Re: junk in *grep* buffers

[Richard Stallman]

Thanks for fixing this.

Re: Fwd: Re: junk in *grep* buffers

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So the patch below should fix the problem because it uses the font-lock-face
> property which is not cleared by font-lock.

This is still not enough when there are multiple matches on the same line
close to each other.  Since we delete some text during fontification
font-lock-fontify-keywords-region will skip over some of the following
text when it readjusts point.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Fwd: Re: junk in *grep* buffers

[Richard Stallman]

    This is still not enough when there are multiple matches on the same line
    close to each other.  Since we delete some text during fontification
    font-lock-fontify-keywords-region will skip over some of the following
    text when it readjusts point.

Is it possible to adjust the position at which
font-lock-fontify-keywords-region will set point, to compensate for
the deletion?  Could the code use a marker to maintain that position?
Then it would be relocated automatically.

Re: Fwd: Re: junk in *grep* buffers

[Juri Linkov]

>     This is still not enough when there are multiple matches on the same line
>     close to each other.  Since we delete some text during fontification
>     font-lock-fontify-keywords-region will skip over some of the following
>     text when it readjusts point.
>
> Is it possible to adjust the position at which
> font-lock-fontify-keywords-region will set point, to compensate for
> the deletion?  Could the code use a marker to maintain that position?
> Then it would be relocated automatically.

The following hack fixes the problem, but this is an imperfect solution.
It sets the local variable `pos' from `font-lock-fontify-keywords-region'
to avoid changing the current position on the line:

	      ;; Ensure forward progress.
	      (if (< (point) pos) (goto-char pos))

Index: lisp/progmodes/grep.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/progmodes/grep.el,v
retrieving revision 1.36
diff -u -r1.36 grep.el
--- lisp/progmodes/grep.el	7 May 2005 16:21:12 -0000	1.36
+++ lisp/progmodes/grep.el	9 May 2005 22:16:22 -0000
@@ -303,7 +303,8 @@
 	 ;; Delete markers with `replace-match' because it updates
 	 ;; the match-data, whereas `delete-region' would render it obsolete.
 	 (replace-match "" t t nil 3)
-	 (replace-match "" t t nil 1)))))
+	 (replace-match "" t t nil 1)
+	 (setq pos (point))))))
    "Additional things to highlight in grep output.
 This gets tacked on the end of the generated expressions.")

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: [Lute.Kamstra.lists@xs4all.nl: Re: junk in *grep* buffers]

[Kai Großjohann]

Richard Stallman <rms@gnu.org> writes:

> Would someone please investigate this bug?
> Please respond to this message if you investigate it.

Lute Kamstra writes:

> I get a *grep* buffer with occurrences of "(define-minor-mode" in lisp
> files.  Every time that I tried, most lines in this buffer give the
> text "(define-minor-mode" the grep-match-face, but a few lines don't
> fontify "(define-minor-mode".  The strange thing is that the lines
> that don't fontify "(define-minor-mode" are different every time I
> invoke grep.

Very wild guess.  Please do C-h v tramp-chunksize RET and run the Lisp
code shown in the documentation for tramp-chunksize and report your
findings.

The Lisp snippet contains a number, 1000.  If you do not observe a
problem (i.e., you that bytes sent always equals bytes received), then
please also try larger values, 10000 or 100000 or 1000000.

Kai

Re: Fwd: Re: junk in *grep* buffers

[Richard Stallman]

    The following hack fixes the problem, but this is an imperfect solution.
    It sets the local variable `pos' from `font-lock-fontify-keywords-region'
    to avoid changing the current position on the line:

Could the code in font-lock-fontify-keywords-region use a marker to
maintain that position?  Then it would be relocated automatically.

Re: Fwd: Re: junk in *grep* buffers

[Stefan Monnier]

>     The following hack fixes the problem, but this is an imperfect solution.
>     It sets the local variable `pos' from `font-lock-fontify-keywords-region'
>     to avoid changing the current position on the line:

> Could the code in font-lock-fontify-keywords-region use a marker to
> maintain that position?  Then it would be relocated automatically.

Would the patch below do the trick?


        Stefan


--- font-lock.el	05 mai 2005 16:09:23 -0400	1.245
+++ font-lock.el	10 mai 2005 13:35:07 -0400	
@@ -1405,6 +1405,7 @@
   (let ((case-fold-search font-lock-keywords-case-fold-search)
 	(keywords (cddr font-lock-keywords))
 	(bufname (buffer-name)) (count 0)
+        (pos (make-marker))
 	keyword matcher highlights)
     ;;
     ;; Fontify each item in `font-lock-keywords' from `start' to `end'.
@@ -1439,12 +1440,13 @@
 	(while highlights
 	  (if (numberp (car (car highlights)))
 	      (font-lock-apply-highlight (car highlights))
-	    (let ((pos (point)))
+	    (set-marker pos (point))
 	      (font-lock-fontify-anchored-keywords (car highlights) end)
 	      ;; Ensure forward progress.
-	      (if (< (point) pos) (goto-char pos))))
+            (if (< (point) pos) (goto-char pos)))
 	  (setq highlights (cdr highlights))))
-      (setq keywords (cdr keywords)))))
+      (setq keywords (cdr keywords)))
+    (set-marker pos nil)))
 
 ;;; End of Keyword regexp fontification functions.
 \f

Re: Fwd: Re: junk in *grep* buffers

[Juri Linkov]

>> Could the code in font-lock-fontify-keywords-region use a marker to
>> maintain that position?  Then it would be relocated automatically.
>
> Would the patch below do the trick?

Yes, it fixes the problem of too close multiple matches.

-- 
Juri Linkov
http://www.jurta.org/emacs/