font-lock patch for automated edits

font-lock patch for automated edits

[Eric M. Ludlam]

Hi,

  This patch is for font-lock.el CVS version 1.339.

  I have some tests for the CEDET/Semantic incremental parsing engine.
This test calls 'erase-buffer', and performs edits very quickly, and
was causing font-lock to throw errors in my tests.

  This patch basically protects various determined regions (BEG/END)
from being out of bounds of a buffer.  I'd guess there is a better way
to do it, but this allows my tests to run.

  I also added a couple doc strings as checkdoc was unhappy with this
file.

Thanks
Eric
----------------------------

*** font-lock.el.~1.339.~	2008-06-27 03:34:46.000000000 -0400
--- font-lock.el	2008-08-30 10:24:00.000000000 -0400
***************
*** 1018,1027 ****
--- 1018,1034 ----
    (funcall font-lock-unfontify-buffer-function))
  
  (defun font-lock-fontify-region (beg end &optional loudly)
+   "Fontify the region between BEG and END.
+ Optional argument LOUDLY is passed to the `font-lock-fontify-region-function'."
    (font-lock-set-defaults)
+   (when (< beg (point-min)) (setq beg (point-min)))
+   (when (> end (point-max)) (setq end (point-max)))
    (funcall font-lock-fontify-region-function beg end loudly))
  
  (defun font-lock-unfontify-region (beg end)
+   "Unfontify the region between BEG and END."
+   (when (< beg (point-min)) (setq beg (point-min)))
+   (when (> end (point-max)) (setq end (point-max)))
    (save-buffer-state nil
      (funcall font-lock-unfontify-region-function beg end)))
  
***************
*** 1093,1099 ****
                                 font-lock-beg 'font-lock-multiline)
                                (point-min))))
      ;;
!     (when (get-text-property font-lock-end 'font-lock-multiline)
        (setq changed t)
        (setq font-lock-end (or (text-property-any font-lock-end (point-max)
                                                   'font-lock-multiline nil)
--- 1100,1107 ----
                                 font-lock-beg 'font-lock-multiline)
                                (point-min))))
      ;;
!     (when (and (< font-lock-end (point-max))
! 	       (get-text-property font-lock-end 'font-lock-multiline))
        (setq changed t)
        (setq font-lock-end (or (text-property-any font-lock-end (point-max)
                                                   'font-lock-multiline nil)
***************
*** 1163,1168 ****
--- 1171,1177 ----
  what properties to clear before refontifying a region.")
  
  (defun font-lock-default-unfontify-region (beg end)
+   "Remove text properties between BEG and END."
    (remove-list-of-text-properties
     beg end (append
  	    font-lock-extra-managed-props

Re: font-lock patch for automated edits

[Stefan Monnier]

>   I have some tests for the CEDET/Semantic incremental parsing engine.
> This test calls 'erase-buffer', and performs edits very quickly, and
> was causing font-lock to throw errors in my tests.

Do you have backtraces for those bugs?

>   This patch basically protects various determined regions (BEG/END)
> from being out of bounds of a buffer.  I'd guess there is a better way
> to do it, but this allows my tests to run.

I'm not sure if there's a better place to enforce the bounds, but it's
indeed possible.  Some stack traces would help figure it out,


        Stefan

Re[2]: font-lock patch for automated edits

[Eric M. Ludlam]

>>> Stefan Monnier <monnier@iro.umontreal.ca> seems to think that:
>>   I have some tests for the CEDET/Semantic incremental parsing engine.
>> This test calls 'erase-buffer', and performs edits very quickly, and
>> was causing font-lock to throw errors in my tests.
>
>Do you have backtraces for those bugs?

Hi,

  After fixing the first bug, I'd bump into the next.  I undid my
changes to font-lock, and attached my first stack trace.

  Also see attached a very short version of what I was running.  I
need to run it twice to get the error.  It's actually as easy as just
typing:

M-x erase-buffer RET

  in a buffer with font-locked stuff in it.

It If you want to try out the entire setup I was running, you could
get CEDET from the source-forge CVS, do a build, and then run
semantic-utest-main.

Eric

------------------------
Debugger entered--Lisp error: (args-out-of-range 88 88)
  get-text-property(88 font-lock-multiline)
  font-lock-extend-region-multiline()
  font-lock-default-fontify-region(1 88 nil)
  font-lock-fontify-region(1 88)
  font-lock-after-change-function(1 1 87)
  erase-buffer()
  (let ((buff ...)) (switch-to-buffer buff) (setq buffer-offer-save nil) (erase-buffer) (insert "/* Test file for C language for Unit Tests */\n\n#include <stdio.h>\n#include \"sutest.h\"\n\n") (set-buffer-modified-p nil))
  crash-font-lock()
  call-interactively(crash-font-lock t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)

------------------------

;; Enable global-font-lock mode before using this
(defun crash-font-lock ()
  "Crash font lock by fast-editing a buffer."
  (interactive)
  (let ((buff (find-file-noselect "/tmp/CFL.cpp")))
    (switch-to-buffer buff)
    (setq buffer-offer-save nil)
    (erase-buffer)
    (insert "/* Test file for C language for Unit Tests */

#include <stdio.h>
#include \"sutest.h\"

"
	    )
    (set-buffer-modified-p nil)
    ))
  

Re[2]: font-lock patch for automated edits

[Eric M. Ludlam]

>>> Stefan Monnier <monnier@iro.umontreal.ca> seems to think that:
>>   I have some tests for the CEDET/Semantic incremental parsing engine.
>> This test calls 'erase-buffer', and performs edits very quickly, and
>> was causing font-lock to throw errors in my tests.
>
>Do you have backtraces for those bugs?
>
>>   This patch basically protects various determined regions (BEG/END)
>> from being out of bounds of a buffer.  I'd guess there is a better way
>> to do it, but this allows my tests to run.
>
>I'm not sure if there's a better place to enforce the bounds, but it's
>indeed possible.  Some stack traces would help figure it out,
  [ ... ]

Hello,

  I updated my version of Emacs from CVS recently, and this problem
still occurs.

  I also have a simplified reproduction step.  Start with this C++
file:

----------
/* A comment */
/*
 * A multi-line comment
 */
-----------

Make sure font lock has fontified this entire buffer.

Mark the entirety of the multi-line comment, and use C-w to kill the
region.  Font lock will throw an error and stop auto-updating.

You can also use C-k on the last line in this buffer, to get the same
effect.

Thanks
Eric

-- 
          Eric Ludlam:                       eric@siege-engine.com
   Siege: www.siege-engine.com          Emacs: http://cedet.sourceforge.net

Re: font-lock patch for automated edits

[Stefan Monnier]

>   I updated my version of Emacs from CVS recently, and this problem
> still occurs.  I also have a simplified reproduction step.  Start with
> this C++ file:

> ----------
> /* A comment */
> /*
>  * A multi-line comment
>  */
> -----------

> Make sure font lock has fontified this entire buffer.
> Mark the entirety of the multi-line comment, and use C-w to kill the
> region.  Font lock will throw an error and stop auto-updating.

I see now.  It appears to be a bug in c-extend-after-change-region
(assigned to font-lock-extend-after-change-region-function) which
returns a region that extends past the end of the buffer.

Alan, can you take a look at it?  In the above recipe, it helps to do:
M-: (setq fnt-lock-support-mode nil debug-on-error t) RET and then
toggle font-lock-mode OFF and back ON before doing the C-w.


        Stefan

Re[2]: font-lock patch for automated edits

[Eric M. Ludlam]

>>> Stefan Monnier <monnier@IRO.UMontreal.CA> seems to think that:
>>   I updated my version of Emacs from CVS recently, and this problem
>> still occurs.  I also have a simplified reproduction step.  Start with
>> this C++ file:
>
>> ----------
>> /* A comment */
>> /*
>>  * A multi-line comment
>>  */
>> -----------
>
>> Make sure font lock has fontified this entire buffer.
>> Mark the entirety of the multi-line comment, and use C-w to kill the
>> region.  Font lock will throw an error and stop auto-updating.
>
>I see now.  It appears to be a bug in c-extend-after-change-region
>(assigned to font-lock-extend-after-change-region-function) which
>returns a region that extends past the end of the buffer.
>
>Alan, can you take a look at it?  In the above recipe, it helps to do:
>M-: (setq fnt-lock-support-mode nil debug-on-error t) RET and then
>toggle font-lock-mode OFF and back ON before doing the C-w.

Thanks Stefan.  My .emacs is so old I had old font-lock-support stuff
in it trying to allow functionality between Emacs 20 and 21 and
various support modes that I think has the effect your M-: line above
creates.

Without the old config, thus enabling jit-lock-mode, my automated
editing test suites work fine when font lock is enabled.

Eric

-- 
          Eric Ludlam:                       eric@siege-engine.com
   Siege: www.siege-engine.com          Emacs: http://cedet.sourceforge.net